-- Logs begin at Tue 2022-06-07 12:27:57 UTC, end at Tue 2022-06-07 13:40:50 UTC. -- Jun 07 13:03:04.237873 dv-cinder-844601-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 07 13:03:04.246384 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jun 7 13:03:04 2022] *** Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: nodename: dv-cinder-844601-3-iscsi Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: machine: x86_64 Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: clock source: unix Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: pcre jit disabled Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: detected number of CPU cores: 4 Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: current working directory: / Jun 07 13:03:04.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: detected binary path: /usr/bin/uwsgi-core Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: your processes number limit is 47680 Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: your memory page size is 4096 bytes Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: detected max file descriptor number: 2048 Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: lock engine: pthread robust mutexes Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: thunder lock: enabled Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 07 13:03:04.248856 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 07 13:03:04.293180 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: Python main interpreter initialized at 0x5612225399c0 Jun 07 13:03:04.293180 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: python threads support enabled Jun 07 13:03:04.293180 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: your server socket listen backlog is limited to 100 connections Jun 07 13:03:04.293180 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: your mercy for graceful operations on workers is 90 seconds Jun 07 13:03:04.293520 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: mapped 403077 bytes (393 KB) for 2 cores Jun 07 13:03:04.293601 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: *** Operational MODE: preforking *** Jun 07 13:03:04.293879 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: *** uWSGI is running in multiple interpreter mode *** Jun 07 13:03:04.293879 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: spawned uWSGI master process (pid: 80500) Jun 07 13:03:04.294056 dv-cinder-844601-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 07 13:03:04.294592 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: spawned uWSGI worker 1 (pid: 80506, cores: 1) Jun 07 13:03:04.295420 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: spawned uWSGI worker 2 (pid: 80507, cores: 1) Jun 07 13:03:04.295420 dv-cinder-844601-3-iscsi devstack@keystone.service[80500]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 07 13:03:09.969223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 13:03:09.969223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: from cryptography.utils import int_from_bytes Jun 07 13:03:09.970021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 13:03:09.970021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: from cryptography.utils import int_from_bytes Jun 07 13:03:09.971799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 13:03:09.971799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: from cryptography.utils import int_from_bytes Jun 07 13:03:09.972256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 07 13:03:09.972256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: from cryptography.utils import int_from_bytes Jun 07 13:03:10.019688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 13:03:10.019891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 13:03:10.020088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] command line args: [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 13:03:10.020356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 13:03:10.020532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ================================================================================ {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 13:03:10.020785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] admin_token = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.020869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 07 13:03:10.020990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 07 13:03:10.021069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] config_dir = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.021182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] command line args: [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 07 13:03:10.021260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.021363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] config_source = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.021476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 07 13:03:10.021561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] debug = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.021655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ================================================================================ {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 07 13:03:10.021734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.021965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] admin_token = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] config_dir = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] config_source = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] debug = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.022947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_config_append = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_dir = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_file = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.023894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_config_append = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_dir = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_file = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.024900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.025915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.026913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] publish_errors = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.027997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] publish_errors = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] standard_threads = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.028957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] use_journal = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] use_json = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.029962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] standard_threads = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] use_stderr = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] use_syslog = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] use_journal = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.030970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.031059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] use_json = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.031131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] use_stderr = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.031204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.031288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] use_syslog = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.031453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.031532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 07 13:03:10.031690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.031783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.031940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.032984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.033148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.033224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.033384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.033461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.034942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.035911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.036877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.037968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.038930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.039941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.040873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.041931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.042008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.042159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.042235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.042403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.042580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.042821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.043926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.044921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.045951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.046993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.047870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.048971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.049048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.049255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.049859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.050965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.051924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.052943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.053913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.054947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.055972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.external = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.056947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.external = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.password = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] auth.token = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.057876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.password = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] auth.token = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.058985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.059998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.060946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.061948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.062091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.062221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.062608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.062746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.062894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.063897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.064923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065388 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.065920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.066972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.067906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.068034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.068919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.069975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.070942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.071954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.072999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.073894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.074963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.075892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.076934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.077976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.078905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.079933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.080059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.080184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.080308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.080920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.081962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.082894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.083930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.084979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.085868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.086909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.087932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.088999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.089888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.090983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.091953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092613 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.092875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] role.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.093890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] role.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] role.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.094931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] role.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.095979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.096071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.096736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.096813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.096886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.096959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.097919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098601 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.098953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.099945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.100983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.101880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.102853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.connection = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.103956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.connection = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.104917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.105939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.106932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.107974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.108055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.108128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80506) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 13:03:10.108211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.108438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.108679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.108912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 07 13:03:10.109083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80507) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 07 13:03:10.136487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.136719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.136944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.137303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.137538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.137764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.145689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.146575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.173685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.173913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.174603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.174852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.227281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.228209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.275155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.275387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.275992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.276227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.341382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.341608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.341828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.341973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.342895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.343114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.366976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.367826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.378177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.378477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.390891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.391902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.392018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.392102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.392227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80506) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.392311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.392480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.392693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80507) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 07 13:03:10.410772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.411000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 13:03:10.411085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.411245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 13:03:10.411342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.411526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 13:03:10.411608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.411759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 07 13:03:10.411841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.412783 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.423380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.423658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.423748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.423922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.424009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.424220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.424307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.424528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.432640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.433109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.436051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.436332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.436425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.436614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.436720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.436894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.436986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.437178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.437516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.437691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.437790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.437959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.438859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.439034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.439133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.439297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.466389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.466864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.468108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.468386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.468539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.468814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.473780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.474066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.474196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.474346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.474473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.474748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.481659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.481893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.481991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.482241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.486364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.486649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.486733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.486979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.487894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.488077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.488159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.488339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.488454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.488711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.507590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.507880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.507995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.508267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.512494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.512806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.516267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.516535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.516619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.516825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.522178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.522493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.525804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.526067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.529467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.529693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.529777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.529985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530067 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.530962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.531136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.549896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.550178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.550262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.550446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.550530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.550708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.550791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.551029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.558621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.559027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.561881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.562157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.562498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.562751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.566014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.566290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.566373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.566568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.566652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.566883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.566982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.567172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.567255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.567454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.580516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.581284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.581386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.581567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.581650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.581842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.581925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.582961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.600467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.600685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.603752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.603947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.607218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.607388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.608830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.608964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.610461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.610592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.610736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.610888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.611042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.611176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.618318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.618486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.619976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.620109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.623343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.623429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.623635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.623720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.623924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.624020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.624210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.624295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.633805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.633921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.634084 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.634259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.634369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.634714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.642341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80507) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.642617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.642916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.643755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.644034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.644312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.644588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.644862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.645133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.645411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.645675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.645944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.646224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.646483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80507) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.649736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80506) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 07 13:03:10.650012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.650284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.650552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.650817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.651107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.651385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.651655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.651922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.652186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.652450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.652720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.652989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.653243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80506) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 07 13:03:10.685306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5612225399c0 pid: 80507 (default app) Jun 07 13:03:10.688967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a22ae2-e167-47b5-8e07-3867ba1e02c0 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:10.689256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a22ae2-e167-47b5-8e07-3867ba1e02c0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:10.689534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a22ae2-e167-47b5-8e07-3867ba1e02c0 None None] PATH_INFO: `/v3` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:10.691425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5612225399c0 pid: 80506 (default app) Jun 07 13:03:10.694242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Tue Jun 7 13:03:10 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:18.587962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:18.588509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:18.588509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:18.592842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:18.598812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:18.599245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:18.599399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:18.667105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: INFO dbcounter [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] Registered counter for database keystone Jun 07 13:03:18.688558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG oslo_db.sqlalchemy.engines [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80507) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 13:03:18.695344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writer thread running {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 13:03:18.975167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG passlib.handlers.bcrypt [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80507) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 13:03:18.975511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG passlib.handlers.bcrypt [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80507) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 13:03:19.033935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:19.035610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:19.158295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 13:03:19.321982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:19.391393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: INFO dbcounter [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] Registered counter for database keystone Jun 07 13:03:19.410059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG oslo_db.sqlalchemy.engines [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80506) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 07 13:03:19.416120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writer thread running {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 07 13:03:19.637573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762ec48-b3a9-4976-957b-2ac1097fafa8 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:19.641716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:19.642381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:19.642666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:19.642971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:19.650859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:19.702247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 13:03:19.702586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e597cc7-2a44-4863-acf2-545fc3d3f5b6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:19.711859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Tue Jun 7 13:03:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:03:19.717832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:19.732461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d554fb-e6c6-493b-843e-e115cf95da58 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:19.736803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:19.738495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:19.738781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:19.739093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] PATH_INFO: `/v3/projects/f9260a652cb24f75867436268df3aa69` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:19.740605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] RBAC: Authorizing `identity:get_project(project_id=f9260a652cb24f75867436268df3aa69)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:19.789962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 07 13:03:19.790302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003e7bc-b7c8-488b-8798-5702bd54e852 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:19.796098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Tue Jun 7 13:03:19 2022] GET /identity/v3/projects/f9260a652cb24f75867436268df3aa69 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:22.000795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8b3cf-2cd8-4126-970c-e3314608756c None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:22.001332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8b3cf-2cd8-4126-970c-e3314608756c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:22.001332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8b3cf-2cd8-4126-970c-e3314608756c None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:22.006149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:22.011472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:22.011842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:22.012031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:22.030814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:22.032348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:22.040105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 07 13:03:22.062509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8b3cf-2cd8-4126-970c-e3314608756c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:22.075297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8b3cf-2cd8-4126-970c-e3314608756c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:22.079480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37d9a9-0aa0-4d5a-a6de-c0311a0f8aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:22.080099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37d9a9-0aa0-4d5a-a6de-c0311a0f8aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:22.080384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37d9a9-0aa0-4d5a-a6de-c0311a0f8aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:22.081271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37d9a9-0aa0-4d5a-a6de-c0311a0f8aa1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:22.088860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37d9a9-0aa0-4d5a-a6de-c0311a0f8aa1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:22.090626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37d9a9-0aa0-4d5a-a6de-c0311a0f8aa1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:22.099207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Tue Jun 7 13:03:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 13:03:22.104281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:22.113509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7c3c9-c316-4975-9c70-6db59b05c631 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:22.117819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d26f35-5249-4651-929e-ecebe8bc1c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:22.118667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26f35-5249-4651-929e-ecebe8bc1c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:22.118984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26f35-5249-4651-929e-ecebe8bc1c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:22.119261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26f35-5249-4651-929e-ecebe8bc1c8d admin admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:22.120627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26f35-5249-4651-929e-ecebe8bc1c8d admin admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:22.122418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26f35-5249-4651-929e-ecebe8bc1c8d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:22.123987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Tue Jun 7 13:03:22 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:24.394412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0114e-58eb-43e5-9e39-a2a260f86a5b None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.394987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0114e-58eb-43e5-9e39-a2a260f86a5b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.394987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0114e-58eb-43e5-9e39-a2a260f86a5b None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.396351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:24.401408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.401801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.402085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.419414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:24.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:24.421934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437cfbc-b30e-4708-8489-bf3ebcdacc8a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.422222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437cfbc-b30e-4708-8489-bf3ebcdacc8a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.422501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437cfbc-b30e-4708-8489-bf3ebcdacc8a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.423870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:24.429463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.429748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.430021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.469388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG passlib.handlers.bcrypt [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80506) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 07 13:03:24.469845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG passlib.handlers.bcrypt [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80506) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 07 13:03:24.500532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 6/12] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:24.516869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:24.518470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:24.523944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:24.546773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:24.618128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 7/13] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:24.625944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b517514-2b83-4aa1-a643-baef7f268590 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:24.640741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:24.659081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:24.659816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.660102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.660384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.671361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:24.682994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:24.684106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:24.684931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:03:24.690135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:24.739640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66836fc-c406-45cd-8d9e-ef9fafc3d813 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:24.741931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:24.771915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:24.779564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:24.780469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.780757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.781032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.791133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77520f-20b5-4d0d-b88a-7e527bdbdc5f None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:24.824683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:24.825351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.825637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.825916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.826673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:24.827010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:24.828418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:24.840200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 8/15] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:24.845713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:24.847701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:24.849223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:24.945130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d613-0e13-49ff-a09b-02516d742d48 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:24.951785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 8/16] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:03:24.957145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:24.959873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:24.978337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:24.979237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:24.979518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:24.979790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:24.980983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:24.982437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:24.984069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:24 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:24.989790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:25.003980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:25.056713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8e73-02a0-4a24-8fc0-421b669d7493 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:25.091457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:25.091905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b51e6-ed23-4e78-804a-12d74df9f9bf None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:25.092261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:25.092549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:25.092826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:25.111291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:25.112762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:25.124527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:25.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:25.125585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:25.125861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:25.127049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:25.128501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:25.130095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 10/18] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:03:25.133142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:25.134126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:25.135373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:03:25.135373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:03:25.135373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:03:25.135373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:25.135373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 9/19] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:03:25.135777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:25.140785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:25.196458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:25.216330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:25.235992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3b098-2cc1-46c5-8b06-49f853c452d4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:25.240834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f3588-76d2-44f9-a5c1-4e23d88398c4 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:25.269055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:25.269711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:25.269998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:25.270282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:25.271153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.domain.id=default, user.id=05d0278bf67f4654af88f4f7c25092c2) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:25.271476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:25.272938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaa61c-38bd-47f9-847a-4edc5a76d89d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:25.275227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:25.275879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:25.276165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:25.276442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:25.277167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:25.277469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:25.278823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad70f88-d88f-4bba-8965-7287552deaf9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:25.279625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 11/20] 10.222.0.61 () {56 vars in 1322 bytes} [Tue Jun 7 13:03:25 2022] GET /identity/v3/role_assignments?user.id=05d0278bf67f4654af88f4f7c25092c2&scope.domain.id=default&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:25.298314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 10/21] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:27.764022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cac8c-1353-4108-bd6e-32b87c66b10e None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:27.765071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cac8c-1353-4108-bd6e-32b87c66b10e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:27.765570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cac8c-1353-4108-bd6e-32b87c66b10e None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:27.767565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 12/22] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:27.773855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:27.774383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:27.774383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:27.799131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:27.800409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:27.904931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 11/23] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:27.938711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911cac8c-1353-4108-bd6e-32b87c66b10e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:28.080157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911cac8c-1353-4108-bd6e-32b87c66b10e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:28.128551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275a635-13c8-4329-b3ba-7d97ed6fd039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:28.129715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275a635-13c8-4329-b3ba-7d97ed6fd039 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.130196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275a635-13c8-4329-b3ba-7d97ed6fd039 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.130675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275a635-13c8-4329-b3ba-7d97ed6fd039 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.147832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275a635-13c8-4329-b3ba-7d97ed6fd039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-c275a635-13c8-4329-b3ba-7d97ed6fd039 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:28.155870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:28.156724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:28.157486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 13/24] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:28.167988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:28.168659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555da51e-5a19-45d8-afbe-75cf1c25cecc None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.168952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555da51e-5a19-45d8-afbe-75cf1c25cecc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.169226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555da51e-5a19-45d8-afbe-75cf1c25cecc None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.174952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 14/25] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:28.177587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7fef8-b09d-4b59-9fe5-45c5f19ac8e1 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.177882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7fef8-b09d-4b59-9fe5-45c5f19ac8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.178157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7fef8-b09d-4b59-9fe5-45c5f19ac8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.187535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:28.204498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ffc7fef8-b09d-4b59-9fe5-45c5f19ac8e1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:28.206250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ffc7fef8-b09d-4b59-9fe5-45c5f19ac8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:28.267235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:28.361574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 15/26] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:28.363763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1d4eb-e31d-49fa-b476-ec1733e95201 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.364292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1d4eb-e31d-49fa-b476-ec1733e95201 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.364766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1d4eb-e31d-49fa-b476-ec1733e95201 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.366651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 16/27] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:28.372171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.372725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.373209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.395051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:28.396791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:28.429011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72459b-d044-47e0-a49e-4332ae199d01 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:28.503340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:28.504437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.504839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.505118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.505968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:28.506275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:28.507711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:28.516061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 17/28] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:28.517105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:28.540993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 12/29] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:28.546819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:28.651778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:28.695692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ce5b5-20c6-4c3b-8604-3c5824a5c655 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:28.713835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae598aa-9e8c-4fca-9907-cfc5048d39c9 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:28.727727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:28.734240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:28.735363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.735850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.736323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.748168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:28.749883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:03:28.769489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:28.770805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:28.771631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:03:28.771631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:03:28.771631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:03:28.771631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:28.773655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 18/30] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:28.774680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:28.788985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:28.789920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.790220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.790487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.796905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:28.798544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:28.800934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 13/31] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:28 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:28.801690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:28.887757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c4e3-45b8-4b86-a989-59e6cdd52c10 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:28.906188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:28.925848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98071c8-2ebd-45df-b618-9b1dffe91535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:28.926930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98071c8-2ebd-45df-b618-9b1dffe91535 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:28.927417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98071c8-2ebd-45df-b618-9b1dffe91535 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:28.927908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98071c8-2ebd-45df-b618-9b1dffe91535 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:28.928857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98071c8-2ebd-45df-b618-9b1dffe91535 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:28.930475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98071c8-2ebd-45df-b618-9b1dffe91535 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:28.977384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec23518-c7c8-47fa-956b-f2f5d3122328 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:29.012809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:29.013481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.013773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.014062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.014795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:29.015154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:29.016531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:29.034856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 14/32] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:29.035745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:29.074107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:29.075132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 19/33] 10.222.0.61 () {60 vars in 1131 bytes} [Tue Jun 7 13:03:28 2022] POST /identity/v3/roles => generated 219 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:29.077453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff023c2-8004-444b-972b-ab97a9691afe None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.077948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff023c2-8004-444b-972b-ab97a9691afe None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.078427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff023c2-8004-444b-972b-ab97a9691afe None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.080019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 20/34] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:29.081050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff023c2-8004-444b-972b-ab97a9691afe None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:29.156394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85092935-af23-439c-81fe-9c645e2b46b9 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:29.196591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177af18-1efa-46a5-8ed8-19cc101cdafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:29.197413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177af18-1efa-46a5-8ed8-19cc101cdafb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.197703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177af18-1efa-46a5-8ed8-19cc101cdafb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.197987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177af18-1efa-46a5-8ed8-19cc101cdafb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.199285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177af18-1efa-46a5-8ed8-19cc101cdafb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:29.200967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177af18-1efa-46a5-8ed8-19cc101cdafb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:29.203140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 15/35] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:29.210027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.210317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.210611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.229987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:29.231400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:29.246816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff023c2-8004-444b-972b-ab97a9691afe None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:29.310389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7475c6-548a-4f2f-9881-d9d48ef776c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:29.311545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7475c6-548a-4f2f-9881-d9d48ef776c1 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.312034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7475c6-548a-4f2f-9881-d9d48ef776c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.312523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7475c6-548a-4f2f-9881-d9d48ef776c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.313758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7475c6-548a-4f2f-9881-d9d48ef776c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:29.315266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 16/36] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:29.316009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7475c6-548a-4f2f-9881-d9d48ef776c1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:29.316217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:29.371262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 21/37] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:03:29 2022] POST /identity/v3/projects => generated 301 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:29.372846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65374ac9-0315-4d9c-9a92-30211919d532 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.373152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65374ac9-0315-4d9c-9a92-30211919d532 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.373448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65374ac9-0315-4d9c-9a92-30211919d532 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.377506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:29.379407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 22/38] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:29.380583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65374ac9-0315-4d9c-9a92-30211919d532 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:29.427294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:29.505429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b524e-e7e9-444f-883b-b153123dcc7e None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:29.542890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65374ac9-0315-4d9c-9a92-30211919d532 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:29.549049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb1a30-e81f-401d-a07d-55d89b547469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:29.549321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb1a30-e81f-401d-a07d-55d89b547469 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.550186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb1a30-e81f-401d-a07d-55d89b547469 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.550475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb1a30-e81f-401d-a07d-55d89b547469 None admin] PATH_INFO: `/v3/domains/default/users/05d0278bf67f4654af88f4f7c25092c2/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.553230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb1a30-e81f-401d-a07d-55d89b547469 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=05d0278bf67f4654af88f4f7c25092c2, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:29.554867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb1a30-e81f-401d-a07d-55d89b547469 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:29.593236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d0e4c-d1e9-4916-90d3-a7b59fe9942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:29.593911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0e4c-d1e9-4916-90d3-a7b59fe9942d None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.594247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0e4c-d1e9-4916-90d3-a7b59fe9942d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.594566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0e4c-d1e9-4916-90d3-a7b59fe9942d None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.595418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0e4c-d1e9-4916-90d3-a7b59fe9942d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:29.597453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0e4c-d1e9-4916-90d3-a7b59fe9942d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:29.616091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 23/39] 10.222.0.61 () {60 vars in 1131 bytes} [Tue Jun 7 13:03:29 2022] POST /identity/v3/roles => generated 225 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:29.617396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49c8b9-e245-4ff2-884d-26204efb6693 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.617637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49c8b9-e245-4ff2-884d-26204efb6693 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.617932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49c8b9-e245-4ff2-884d-26204efb6693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.637453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6e49c8b9-e245-4ff2-884d-26204efb6693 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:29.639088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6e49c8b9-e245-4ff2-884d-26204efb6693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:29.692152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 17/40] 10.222.0.61 () {58 vars in 1361 bytes} [Tue Jun 7 13:03:29 2022] PUT /identity/v3/domains/default/users/05d0278bf67f4654af88f4f7c25092c2/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:03:29.729217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df1509-9291-4b87-b98a-a0322b4fcc8f None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.729518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df1509-9291-4b87-b98a-a0322b4fcc8f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.729807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df1509-9291-4b87-b98a-a0322b4fcc8f None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.730466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 24/41] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:29.735262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 18/42] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:29.741337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840850d-9c39-4690-8872-741616d2947f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.741630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840850d-9c39-4690-8872-741616d2947f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.741918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840850d-9c39-4690-8872-741616d2947f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.758325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df1509-9291-4b87-b98a-a0322b4fcc8f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:29.762414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f840850d-9c39-4690-8872-741616d2947f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:29.763703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f840850d-9c39-4690-8872-741616d2947f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:29.779884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:29.856587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 25/43] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:29.887304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840850d-9c39-4690-8872-741616d2947f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:29.899785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:29.910491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df1509-9291-4b87-b98a-a0322b4fcc8f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:29.958776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:29.959407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:29.959689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:29.959968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:29.960738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:29.962259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:29.979707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 19/44] 10.222.0.61 () {60 vars in 1131 bytes} [Tue Jun 7 13:03:29 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:30.019170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840850d-9c39-4690-8872-741616d2947f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:30.058191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee0a9a-34df-4035-87d3-33e9529b4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:30.059038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee0a9a-34df-4035-87d3-33e9529b4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:30.059337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee0a9a-34df-4035-87d3-33e9529b4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:30.059620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee0a9a-34df-4035-87d3-33e9529b4bc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:30.060904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee0a9a-34df-4035-87d3-33e9529b4bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:30.062478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee0a9a-34df-4035-87d3-33e9529b4bc7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:30.064783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 26/45] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:30.070772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:30.183205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc97d97-37bb-4fdb-b0e4-9b13ef0cde6c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:30.219274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538f032-8937-43e9-ac57-2f5d8942eea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:30.219873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f032-8937-43e9-ac57-2f5d8942eea0 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:30.220164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f032-8937-43e9-ac57-2f5d8942eea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:30.220449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f032-8937-43e9-ac57-2f5d8942eea0 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:30.221175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538f032-8937-43e9-ac57-2f5d8942eea0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:30.222703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538f032-8937-43e9-ac57-2f5d8942eea0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:30.251194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 20/46] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:03:30 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:32.016814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2de20-6af6-4037-8073-aefb64a2a323 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.017331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2de20-6af6-4037-8073-aefb64a2a323 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.017489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2de20-6af6-4037-8073-aefb64a2a323 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.018897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 27/47] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:32.024147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.024547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.024841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.036000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:32.042428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:32.044740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:32.111462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 21/48] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:32.135360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2de20-6af6-4037-8073-aefb64a2a323 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.153880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:32.241585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2de20-6af6-4037-8073-aefb64a2a323 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.274199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.274981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.275272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.275554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.285959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:32.293589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:32.294524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:32.295540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 28/49] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:32.301036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.397441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dbeec-8c61-4b66-af07-db23c09d79c5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.432829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.433507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.433798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.434075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.434852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:32.435169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:32.436553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666da1e-92d7-4bb9-b2e2-490e6ad9fb0e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:32.439119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:32.445865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 22/50] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:32.451287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.490417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff59dbc-daa0-4bfd-9dc5-f3386b4f4911 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.490713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff59dbc-daa0-4bfd-9dc5-f3386b4f4911 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.491062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff59dbc-daa0-4bfd-9dc5-f3386b4f4911 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.492436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 23/51] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:32.497423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.497718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.498005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.514186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:32.515419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:32.528225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:32.551984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd06022-3226-4c94-a6ad-e199f57c3b4b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.582681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 24/52] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:32.584862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.585733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.586022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.586303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.587513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:32.588983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:32.590569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 29/53] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:32 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:32.595851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.606094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.693909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02879e74-b57c-46c1-9a1d-f1f98c3beeee None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.703901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38105d8-1be0-46cf-bf25-7d185a7709de None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.719714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:32.728279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.729074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.729361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.729639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.730811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:32.732277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:32.733887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 25/54] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:32.737878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.738664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.738981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.739259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.739605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.740424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:32.741847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:32.743504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 30/55] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:32.748597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:32.762788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:32.838123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d0413-fafd-4384-b1c5-4e988fd56749 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.850494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a5427-1fe2-4dfa-b08a-3d50061487ff None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:32.871494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.872148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.872436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.872717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.873551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.domain.id=default, user.id=05d0278bf67f4654af88f4f7c25092c2) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:32.873861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:32.875365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833fd91-c54f-4233-ae3f-78e5bccaa76b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:32.883441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 26/56] 10.222.0.61 () {56 vars in 1322 bytes} [Tue Jun 7 13:03:32 2022] GET /identity/v3/role_assignments?user.id=05d0278bf67f4654af88f4f7c25092c2&scope.domain.id=default&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:32.884294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fb449-c5dd-4a7a-9c3a-3cdbfa14e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:32.884958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb449-c5dd-4a7a-9c3a-3cdbfa14e6f5 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:32.885248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb449-c5dd-4a7a-9c3a-3cdbfa14e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:32.885532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fb449-c5dd-4a7a-9c3a-3cdbfa14e6f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:32.886240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fb449-c5dd-4a7a-9c3a-3cdbfa14e6f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:32.887723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fb449-c5dd-4a7a-9c3a-3cdbfa14e6f5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:32.915939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 31/57] 10.222.0.61 () {60 vars in 1140 bytes} [Tue Jun 7 13:03:32 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:03:35.147839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ffaf1-22ac-4237-bbf1-f31fa56b7f53 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:35.148382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ffaf1-22ac-4237-bbf1-f31fa56b7f53 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:35.148382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ffaf1-22ac-4237-bbf1-f31fa56b7f53 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:35.149762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 27/58] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:35.154803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:35.155185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:35.155399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:35.172259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:35.173465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:35.232089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:35.240563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 32/59] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:35.264899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ffaf1-22ac-4237-bbf1-f31fa56b7f53 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:35.323976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:35.364620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ffaf1-22ac-4237-bbf1-f31fa56b7f53 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:35.396778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7af8c-3082-44d1-baf5-5a44c90754cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:35.397559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7af8c-3082-44d1-baf5-5a44c90754cb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:35.397898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7af8c-3082-44d1-baf5-5a44c90754cb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:35.398127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7af8c-3082-44d1-baf5-5a44c90754cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:35.399317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7af8c-3082-44d1-baf5-5a44c90754cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:35.400765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7af8c-3082-44d1-baf5-5a44c90754cb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:35.402343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 28/60] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:35.407575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:35.503676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec38a6-cbb2-4e73-a0f8-bb2708dd4b24 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:35.536414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78058e46-2646-4e01-a318-b277f159d9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:35.537073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78058e46-2646-4e01-a318-b277f159d9ee None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:35.537361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78058e46-2646-4e01-a318-b277f159d9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:35.537643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78058e46-2646-4e01-a318-b277f159d9ee None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:35.538346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78058e46-2646-4e01-a318-b277f159d9ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:35.539828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78058e46-2646-4e01-a318-b277f159d9ee None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:35.564697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 33/61] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:03:35 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:38.680577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45421b-d4d3-4ae8-aed9-47a79b177941 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:38.680577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45421b-d4d3-4ae8-aed9-47a79b177941 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:38.680577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45421b-d4d3-4ae8-aed9-47a79b177941 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:38.681440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 29/62] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:38.687107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:38.687503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:38.687668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:38.711133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:38.711318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0594c0-3f4b-4887-a928-c5fa2d5423a8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:38.711439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0594c0-3f4b-4887-a928-c5fa2d5423a8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:38.711708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0594c0-3f4b-4887-a928-c5fa2d5423a8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:38.712946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:38.713062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 30/63] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:38.725921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:38.726454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:38.726954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:38.747748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:38.749390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:38.795187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:38.837856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 34/64] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:38.865255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 31/65] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:38.870520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:38.889295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:38.983227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.055723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94437595-affd-499a-bb67-a9481c0e78a3 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.071259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290cf07-b527-427a-8d9c-825c2ea64162 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.120505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.121322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.122058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.122350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.134338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:39.136592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.137413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.137704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.137988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:39.142814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:39.143793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:39.145199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 35/66] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:39.150596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.180156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:39.197644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:39.199761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:39.201590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:39.206868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 32/67] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:39.206868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.211968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.323156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b73dc-3804-4be6-a4ba-31dbf15cf0ff None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.327370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00361b60-3ec0-4976-9a12-394d234784ad None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.341631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.360116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.360799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.361095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.361379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.362691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:39.363047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:39.364524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82c0be-1008-44c4-b381-75f7411d79f6 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:39.374251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 33/68] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:39.375787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35329af-ecda-4938-a355-1f173c4c9b02 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.376155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35329af-ecda-4938-a355-1f173c4c9b02 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.376445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35329af-ecda-4938-a355-1f173c4c9b02 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.377758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 34/69] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:39.382101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35329af-ecda-4938-a355-1f173c4c9b02 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.383971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.384661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.384950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.385233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.386004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:39.386314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:39.389347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784e0b-305d-4c54-8c26-154e0229cfa9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:39.397856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 36/70] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:39.399363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.399830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.400283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.428477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:39.428674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:39.504731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 37/71] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:39.505634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.534999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35329af-ecda-4938-a355-1f173c4c9b02 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.536884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.575156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089607cf-3c8c-45fe-b27b-64c288059eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.576077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089607cf-3c8c-45fe-b27b-64c288059eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.576362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089607cf-3c8c-45fe-b27b-64c288059eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.576638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089607cf-3c8c-45fe-b27b-64c288059eb0 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.593083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089607cf-3c8c-45fe-b27b-64c288059eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:39.594534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089607cf-3c8c-45fe-b27b-64c288059eb0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:39.596289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 35/72] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:39.597566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750ff-bf08-4abb-a2a5-038d04b3b29a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.597880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750ff-bf08-4abb-a2a5-038d04b3b29a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.598158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750ff-bf08-4abb-a2a5-038d04b3b29a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.599593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 36/73] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:39.603742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750ff-bf08-4abb-a2a5-038d04b3b29a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.621070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279792bf-26f1-4579-98c0-9bb5f1ce7394 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.647372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.657525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.658402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.658684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.659031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.660210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:39.661745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:39.667962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 38/74] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:39.667962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.716151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750ff-bf08-4abb-a2a5-038d04b3b29a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.755772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4d1e5-218f-417d-9f54-cb2acd71c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.756568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4d1e5-218f-417d-9f54-cb2acd71c279 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.756853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4d1e5-218f-417d-9f54-cb2acd71c279 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.757130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4d1e5-218f-417d-9f54-cb2acd71c279 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.767855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4d1e5-218f-417d-9f54-cb2acd71c279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:39.770943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.774089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e6d3a-852c-4b94-b1c5-6d189ba65c32 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-02e4d1e5-218f-417d-9f54-cb2acd71c279 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:39.776963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:39.777810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:39.778578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 37/75] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:39.779662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.779662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.779958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.797326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:39.798730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:39.814889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:39.816135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:39.816421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:39.816696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:39.870988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 38/76] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:39.871707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:39.900707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:39.902190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:39.902648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:39.973638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe245aaf-da7f-47d1-a0d2-4007426482a7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:39.996831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.005791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 39/77] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:40.005791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.010810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.011485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.011781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.012047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.078374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.079871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.100398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa1bd0-f16f-464d-b169-3a411b25195d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.106960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.134075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.134669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.134993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.135278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.136033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:40.136345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.137731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154fbb4f-bb45-4572-a043-488453915d3b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.146311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 40/78] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.147525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76a34-4250-46e4-baca-6d13705378af None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.147823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76a34-4250-46e4-baca-6d13705378af None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.148108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76a34-4250-46e4-baca-6d13705378af None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.149340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 41/79] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:40.150049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76a34-4250-46e4-baca-6d13705378af None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.185244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 39/80] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:40.186459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.212292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.250902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76a34-4250-46e4-baca-6d13705378af None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.283212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.283598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84f84c-23f2-4092-ac9f-7771f491b152 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.284034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.284321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.284598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.294508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:40.303052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:40.303911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:40.304866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 42/81] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:40.305749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.312580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.321540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365b515-f0d0-4a72-b9dc-568f420f28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.322399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365b515-f0d0-4a72-b9dc-568f420f28ad None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.322682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365b515-f0d0-4a72-b9dc-568f420f28ad None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.322995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365b515-f0d0-4a72-b9dc-568f420f28ad None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.324171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365b515-f0d0-4a72-b9dc-568f420f28ad None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.325615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365b515-f0d0-4a72-b9dc-568f420f28ad None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.327327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 40/82] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.328449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.328733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.329012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.344697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:40.345869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:40.406453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c66d9-40be-4c34-a959-a3f983d46e24 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.410805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 41/83] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:40.411635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.438050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.450430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.451418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.451708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.451987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.453184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.454661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.456517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 43/84] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.457112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.519766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725b019-4bc7-466f-8691-8eae0f7d4e1a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.544737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.553958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d393ffe-c8cf-45a3-92e5-88910b790116 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.554446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.555334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.555620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.555899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.557075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.558530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.560246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 42/85] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.561005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.588086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.588748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.589032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.589306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.590053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:40.590361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.591770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.600189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 44/86] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.600973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.651567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.663283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184aa18-cd59-44b4-ab3e-4ef73379d1a8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.697648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.698294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.698582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.698878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.699715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=1d6906ac8c4644eeb5bc8ca0a350e99c) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:40.700028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.700112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64b60-da44-4329-86be-36f92af3b4c8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.701482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.707635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 43/87] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/role_assignments?user.id=1d6906ac8c4644eeb5bc8ca0a350e99c&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.708450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.740818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.742006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.743014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.743337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.744677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.746187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.748139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 45/88] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.748754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.788508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.838412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c742837-a2e0-4714-809f-b8741cf1edbb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.865429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc02b0-ae4a-41c5-8af8-9cbb2987775c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:40.877560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.878300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.878587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.878903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.890964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:40.909868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:40.910941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:40.911232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:40.911636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:40.912497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=05d0278bf67f4654af88f4f7c25092c2) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:40.912815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.914367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:40.920140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 46/89] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/role_assignments?user.id=05d0278bf67f4654af88f4f7c25092c2&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:40.923076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:40.969463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:40.971075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.042633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:41.093421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 44/90] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:41.094294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac5337-5022-4ef6-9a25-fcd46c647a2b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:41.099528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:41.136950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:41.137863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:41.138147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:41.138427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:41.139837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:41.141422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.143837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 47/91] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:40 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:41.144561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:41.210295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:41.230796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6b3a9-e733-42ea-9e8e-0cd1150a7fcf None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:41.264142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fa385-c222-4f51-8f63-551c8f226005 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:41.282061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:41.283095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:41.283415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:41.284098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:41.284830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:41.286819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.300981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:41.301901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:41.302188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:41.302465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] PATH_INFO: `/v3/projects/02b54d1a76e746ee96a3752099697e08` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:41.303804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] RBAC: Authorizing `identity:get_project(project_id=02b54d1a76e746ee96a3752099697e08)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:41.305405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.307745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 48/92] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:41 2022] GET /identity/v3/projects/02b54d1a76e746ee96a3752099697e08 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:41.308709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:41.323363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 45/93] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:03:41 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:41.324230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:41.355827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:41.489718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64058ce4-7e3f-42aa-bbf8-0307f94f7462 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:41.494029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa2ca-d208-4c42-8f4e-1907083715cc None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:41.532401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:41.533368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:41.533661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:41.534479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:41.535725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=02b54d1a76e746ee96a3752099697e08, user.id=1d6906ac8c4644eeb5bc8ca0a350e99c) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:41.536177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:41.537689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.544401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 46/94] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:41 2022] GET /identity/v3/role_assignments?user.id=1d6906ac8c4644eeb5bc8ca0a350e99c&scope.project.id=02b54d1a76e746ee96a3752099697e08&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:41.567855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:41.568527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:41.568812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:41.569091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:41.583371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:41.658588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:41.660238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.771009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 49/95] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:41.779340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:41.870390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:41.896527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b5c3d-e3cb-4f9d-8bc4-0cfebbd5889a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:41.933113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23ba48-43ba-46b1-a8dc-a75207318941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:41.933976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ba48-43ba-46b1-a8dc-a75207318941 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:41.934261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ba48-43ba-46b1-a8dc-a75207318941 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:41.934539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ba48-43ba-46b1-a8dc-a75207318941 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:41.935760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23ba48-43ba-46b1-a8dc-a75207318941 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:41.937217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23ba48-43ba-46b1-a8dc-a75207318941 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:41.939519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 47/96] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:41 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:41.946054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:41.970222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:42.056547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be002f8f-0445-4a5f-9e51-780da031d8ca None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:42.096239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:42.096936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:42.097223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:42.097501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:42.098334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] RBAC: Adding query filter params (role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=1d6906ac8c4644eeb5bc8ca0a350e99c) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:42.098646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:42.100143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef031a-a620-433e-b5b7-a748f7045ea7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:42.106692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 50/97] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:41 2022] GET /identity/v3/role_assignments?user.id=1d6906ac8c4644eeb5bc8ca0a350e99c&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:43.197516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ab86d-c916-4669-92fd-3bed7167ebfd None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.198210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ab86d-c916-4669-92fd-3bed7167ebfd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.198210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ab86d-c916-4669-92fd-3bed7167ebfd None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:43.199618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 48/98] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:43.206963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.207830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.208333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:43.229830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:43.231402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:43.304620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 51/99] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:43.339281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ab86d-c916-4669-92fd-3bed7167ebfd None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:43.466595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ab86d-c916-4669-92fd-3bed7167ebfd None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:43.512546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:43.513748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.514255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.514741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:43.536200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:43.544738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:43.545688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:43.546449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:43.549119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 49/100] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:43.555893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:43.575818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:43.682362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788c1fe-bf9e-44a7-bdc7-fbe1bb1914b8 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:43.724227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:43.724830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.725120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.725405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:43.726166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:43.726476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:43.727888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:43.742927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 52/101] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:43.748896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:43.758471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:43.875877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16bf0a-cf5b-4c25-a122-eae0aa2c509e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:43.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a76d2f-bcd0-4b18-9c59-78ec7c1c17b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:43.918602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a76d2f-bcd0-4b18-9c59-78ec7c1c17b4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.918912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a76d2f-bcd0-4b18-9c59-78ec7c1c17b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.919198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a76d2f-bcd0-4b18-9c59-78ec7c1c17b4 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:43.920414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a76d2f-bcd0-4b18-9c59-78ec7c1c17b4 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:43.921887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a76d2f-bcd0-4b18-9c59-78ec7c1c17b4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:43.923647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 50/102] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:43 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:43.929150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:43.975368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6417ce-c617-4aaa-84ab-43d90b9cb269 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.975660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6417ce-c617-4aaa-84ab-43d90b9cb269 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.975856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6417ce-c617-4aaa-84ab-43d90b9cb269 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:43.977539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 51/103] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:43.990131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:43.990421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:43.990706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.020556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:44.021769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:44.061478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:44.098669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043ff04-c7b7-42c3-a03c-3098eb7c54e8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:44.155610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 52/104] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:44.161762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:44.162664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.162982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.163264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.164567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:44.166138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:44.168173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 53/105] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:43 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:44.173779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:44.187779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:44.200328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:44.298855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3719cca-3799-471e-9f19-82e92f8a001c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:44.333771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:44.334964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.335415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.335838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/1d6906ac8c4644eeb5bc8ca0a350e99c/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.339478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] RBAC: Authorizing `identity:create_grant(project_id=33a909cd233e4d5abcb4773bc2a46e27, user_id=1d6906ac8c4644eeb5bc8ca0a350e99c, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:44.341072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:44.359540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 53/106] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:03:44 2022] PUT /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/1d6906ac8c4644eeb5bc8ca0a350e99c/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:03:44.382594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cfaf1-bd99-4bd6-b7f5-300c3886286a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:44.435659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:44.436492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.436777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.437052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.457139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:44.465020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:44.465970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:44.466780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:44.471309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 54/107] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:03:44.477065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:44.559406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:44.614920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d66b-c30f-4dd0-b049-7f008b0a094b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:44.652210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:44.653280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.653772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.654248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.655282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:44.655817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:44.657463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc9f08-7da9-4d88-85c0-7c3f9a76af63 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:44.667385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 54/108] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:44.676283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:44.686907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:44.739643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3349dce-a5d4-440e-a1fd-50be5eafd302 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.740301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3349dce-a5d4-440e-a1fd-50be5eafd302 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.740779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3349dce-a5d4-440e-a1fd-50be5eafd302 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.742690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 55/109] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:44.748264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.748904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.749413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.775461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:44.777032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:44.809141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ffc2e-9627-4dff-b265-16aa0f4e0cdb None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:44.849950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:44.850751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:44.851062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:44.851340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:44.852591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:44.854051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:44.859799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 55/110] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:44 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:44.865199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:44.869953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 56/111] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:44.894260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:44.954617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:44.954991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:44.990861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ca928-8143-40b7-9726-ea216a23cc96 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.002472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900a03c-f2a4-4640-a2e2-4dec485913a9 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.038558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.039455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.039585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.039775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.040011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.040395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.040685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.041018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.041207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:45.042674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:45.044574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 56/112] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:44 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:45.051753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:45.053018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:45.060977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:45.061825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:45.062604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 57/113] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:45.068424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:45.175666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2cc98-0c4e-4b11-b31f-06398fa0d0b3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.193736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:45.213531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.214197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.214665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.215334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.216282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:45.216773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:45.218003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfc45f-1656-4a70-8cef-c4eab0781f7c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.218553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:45.229649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 58/114] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:45.239561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:45.286346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c225b0-2a00-470b-a1e9-93e0a2e06166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.290312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c225b0-2a00-470b-a1e9-93e0a2e06166 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.290815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c225b0-2a00-470b-a1e9-93e0a2e06166 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.291301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c225b0-2a00-470b-a1e9-93e0a2e06166 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/05d0278bf67f4654af88f4f7c25092c2/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.295524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c225b0-2a00-470b-a1e9-93e0a2e06166 None admin] RBAC: Authorizing `identity:create_grant(project_id=33a909cd233e4d5abcb4773bc2a46e27, user_id=05d0278bf67f4654af88f4f7c25092c2, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:45.299257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c225b0-2a00-470b-a1e9-93e0a2e06166 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:45.320699 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:45.326462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 57/115] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:03:45 2022] PUT /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/05d0278bf67f4654af88f4f7c25092c2/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:03:45.328695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6a22f-8756-4b6e-bae9-108a68607d38 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.329358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6a22f-8756-4b6e-bae9-108a68607d38 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.330216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6a22f-8756-4b6e-bae9-108a68607d38 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.332964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 58/116] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:45.339400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.340033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.340502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.383034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:45.388202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:45.452171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110c2f7-f342-4bf6-8f96-5d598d75342a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.491471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.492819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.493370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.493846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.495366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:45.497039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:45.498983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 59/117] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:45 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:45.509057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:45.545559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 59/118] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:45.584678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:45.593100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:45.708494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4ea0a-f5bb-43d7-b359-9237e18b7af4 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.749024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.750257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.750986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.752324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] PATH_INFO: `/v3/projects/02b54d1a76e746ee96a3752099697e08` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.756562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] RBAC: Authorizing `identity:get_project(project_id=02b54d1a76e746ee96a3752099697e08)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:45.760549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:45.763153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 60/119] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:45 2022] GET /identity/v3/projects/02b54d1a76e746ee96a3752099697e08 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:45.771818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:45.795772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ccec7-51f7-476e-bda2-7b2a54329323 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.826888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:45.833319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94374488-6e4c-420e-9203-af66496f5437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.834124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94374488-6e4c-420e-9203-af66496f5437 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.834407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94374488-6e4c-420e-9203-af66496f5437 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.834681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94374488-6e4c-420e-9203-af66496f5437 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.845486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94374488-6e4c-420e-9203-af66496f5437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-94374488-6e4c-420e-9203-af66496f5437 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:45.854039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:45.855197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:45.856039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 60/120] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:03:45.856675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95501286-08c7-4c9a-9981-ec37e469ccf6 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.857121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95501286-08c7-4c9a-9981-ec37e469ccf6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.857213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95501286-08c7-4c9a-9981-ec37e469ccf6 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.858655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 61/121] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:45.865221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90f217-43f6-406b-aa7a-3060199abfd9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.867153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90f217-43f6-406b-aa7a-3060199abfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.868007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90f217-43f6-406b-aa7a-3060199abfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.890370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0b90f217-43f6-406b-aa7a-3060199abfd9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:45.892216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0b90f217-43f6-406b-aa7a-3060199abfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:45.900969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3c98c-68e4-4dcb-b7aa-4430237c73a7 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:45.931624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:45.973081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:45.973901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:45.974188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:45.974468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] PATH_INFO: `/v3/projects/02b54d1a76e746ee96a3752099697e08/users/1d6906ac8c4644eeb5bc8ca0a350e99c/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:45.977802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] RBAC: Authorizing `identity:create_grant(project_id=02b54d1a76e746ee96a3752099697e08, user_id=1d6906ac8c4644eeb5bc8ca0a350e99c, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:45.979452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:46.004779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 61/122] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:03:45 2022] PUT /identity/v3/projects/02b54d1a76e746ee96a3752099697e08/users/1d6906ac8c4644eeb5bc8ca0a350e99c/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:03:46.005713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.034989 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 62/123] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:46.067738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc12f0-437c-452b-82ef-7747386d3be7 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.068086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc12f0-437c-452b-82ef-7747386d3be7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.068378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc12f0-437c-452b-82ef-7747386d3be7 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.069829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 63/124] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:46.083219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc12f0-437c-452b-82ef-7747386d3be7 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.143726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc52572-54eb-4f41-8749-d3fda1cc4be5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.182218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.183007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.183364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.183793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.184691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:46.185005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:46.186441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ace2d-423b-49b6-b6ab-7bc8947442e8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:46.197388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 62/125] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:46.198590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.198911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.199198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.216725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:46.218058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:46.254463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:46.319930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc12f0-437c-452b-82ef-7747386d3be7 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.333094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 63/126] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:46.334026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.366093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:46.385593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.386417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.386700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.387044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.400883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:46.411457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:46.413308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 64/127] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:03:46.414640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.460486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5a5a5-2d20-497e-a04c-b9e8d13f830c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.496759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.497639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.497922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.498199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] PATH_INFO: `/v3/users/728ba87f8e3a45838c2f69e43ee6d0ad` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.512345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] RBAC: Authorizing `identity:get_user(user_id=728ba87f8e3a45838c2f69e43ee6d0ad)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:46.513748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:46.515542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 64/128] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/users/728ba87f8e3a45838c2f69e43ee6d0ad => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:46.516309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.551384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:46.615155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4222fa3-272b-4559-b7f4-f1deb24eb32e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.678769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16be59-a4e2-4d7f-a938-1bbd8c90d027 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.683512 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.684551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.685056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.685554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.689899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:46.691665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:46.693521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 65/129] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:46.694497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.716831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.717507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.717792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.718782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.719821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:46.720130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:46.722014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:46.734182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 65/130] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:46.736364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:46.792581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:46.834530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:46.851946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e73e22-15dd-41d7-a7ab-9a12056637ce None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.861515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61704573-a3ba-4536-9b29-9b86facbd855 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:46.920220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.921576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.922583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.923578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.924932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:46.926913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:46.928481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:46.929352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:46.929833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:46.931353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:46.977517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 66/131] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:03:46 2022] POST /identity/v3/projects => generated 302 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:46.981740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:47.108109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:47.109575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:47.142483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869a5b-7e81-498e-a2f8-30dad90aff0e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:47.182759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:47.184179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.184701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.185171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.186951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:47.188817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:47.191920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 67/132] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:47.197502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:47.208965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:47.216303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:47.303726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 66/133] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:47.309440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:47.434969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6a264-a74d-4a02-babf-748aac94bd90 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:47.501956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b616adf-2c34-4294-8fc0-f3e5b32118d5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:47.507517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:47.508388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.508674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.508952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.514204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:47.515701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:47.517806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 68/134] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:47 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:47.523452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:47.568061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:47.568933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.569221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.569500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.571902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:47.573369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:47.577919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 67/135] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:47 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:47.584765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:47.619187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:47.675337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:47.732384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63691-093c-4aff-9109-55a7f11ff9d3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:47.746722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddde91c-580b-49e5-ad47-a6ac05c54f90 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:47.786084 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:47.786753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.787734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.788022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.789175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] RBAC: Adding query filter params (role.id=f6ce35b028e548f68db97d1f5bfc955e, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=728ba87f8e3a45838c2f69e43ee6d0ad) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:47.789507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:47.791014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9f06f-ea49-4a49-b4c2-8d47d079e4ff None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:47.797070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 68/136] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:47 2022] GET /identity/v3/role_assignments?user.id=728ba87f8e3a45838c2f69e43ee6d0ad&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=f6ce35b028e548f68db97d1f5bfc955e => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:47.798325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee896bd-d39e-44e5-bada-fd93bd6b63ca None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.798618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee896bd-d39e-44e5-bada-fd93bd6b63ca None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.803188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee896bd-d39e-44e5-bada-fd93bd6b63ca None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.804805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 69/137] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:47.810656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.810981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.811250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.816285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:47.817198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:47.817545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:47.817824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/1d6906ac8c4644eeb5bc8ca0a350e99c/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:47.824770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] RBAC: Authorizing `identity:create_grant(project_id=33a909cd233e4d5abcb4773bc2a46e27, user_id=1d6906ac8c4644eeb5bc8ca0a350e99c, role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:47.826240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:47.840821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:47.842061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:47.864878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 69/138] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:03:47 2022] PUT /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/1d6906ac8c4644eeb5bc8ca0a350e99c/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:03:47.944992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 70/139] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:47.989080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:48.107303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccdf32-0637-43bb-af44-bc35e0650370 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:48.145825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:48.146632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:48.146946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:48.147223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:48.149196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:48.159357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:48.167967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:48.168893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:48.169649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:48.170227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 70/140] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:48.178612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:48.200981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:48.313129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cf1ae-6ffa-4b24-b70e-4eab3899d9f6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:48.353977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:48.354662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:48.354975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:48.355262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:48.356027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:48.356338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:48.357739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:48.367552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 71/141] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:48.372849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:48.508481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d453201-4f76-4473-95ef-b0d414585a47 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:48.548592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a7387-ea74-413d-b59e-fc45c436de49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:48.549532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a7387-ea74-413d-b59e-fc45c436de49 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:48.549860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a7387-ea74-413d-b59e-fc45c436de49 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:48.550141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a7387-ea74-413d-b59e-fc45c436de49 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:48.551375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a7387-ea74-413d-b59e-fc45c436de49 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:48.552830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a7387-ea74-413d-b59e-fc45c436de49 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:48.554474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 71/142] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:48 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:48.559934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:48.612820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4846f8-8cef-4083-b920-308f85f83fe9 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:48.613094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4846f8-8cef-4083-b920-308f85f83fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:48.613299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4846f8-8cef-4083-b920-308f85f83fe9 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:48.614949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 72/143] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:48.619951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:48.620255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:48.620539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:48.655828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:48.657025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:48.660206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:48.704644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcee815-3b2d-43d4-806b-97a5bec930b9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:48.767596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:48.768269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:48.768612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:48.768895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:48.770865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:48.789022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 73/144] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:48.832347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:48.886812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:48.888227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:49.052115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:49.116979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701082c-9414-4d99-82b6-29c76e6fb6e6 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:49.128008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 72/145] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:49.134249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:49.189338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:49.190524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.194872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.194872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.207387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:49.216295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:49.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:49.217931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 74/146] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:49.224834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:49.327201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:49.409939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c03503-2c0a-4da2-a7d0-ac295763ebde None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:49.461246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332b21d-0e5a-4181-a5c8-8b3b9d0b84e6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:49.471479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:49.472674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.472956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.473228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.473979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:49.474279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:49.475688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46cf589-02ef-47a3-b93a-9470d08f47c0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:49.490740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 75/147] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:49.491881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c08c-b9bf-4c0b-be58-92aead11fffb None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.492171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c08c-b9bf-4c0b-be58-92aead11fffb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.492448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303c08c-b9bf-4c0b-be58-92aead11fffb None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.493810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 76/148] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:49.503498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c08c-b9bf-4c0b-be58-92aead11fffb None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:49.530184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a79d78-f33b-46eb-882d-bd9d9547dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:49.531075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a79d78-f33b-46eb-882d-bd9d9547dfbe None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.531355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a79d78-f33b-46eb-882d-bd9d9547dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.531623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a79d78-f33b-46eb-882d-bd9d9547dfbe None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.534797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a79d78-f33b-46eb-882d-bd9d9547dfbe None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:49.536268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a79d78-f33b-46eb-882d-bd9d9547dfbe None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:49.537975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 73/149] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:49 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:49.544313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.546972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.547255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.578488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:49.579710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:49.652021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:49.704861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303c08c-b9bf-4c0b-be58-92aead11fffb None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:49.716656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 74/150] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:49.717278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:49.740291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:49.782744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:49.783654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.784920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.785203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] PATH_INFO: `/v3/users/05d0278bf67f4654af88f4f7c25092c2` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.787556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] RBAC: Authorizing `identity:get_user(user_id=05d0278bf67f4654af88f4f7c25092c2)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:49.789142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:49.790891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 77/151] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:49 2022] GET /identity/v3/users/05d0278bf67f4654af88f4f7c25092c2 => generated 247 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:49.791655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:49.895947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a1e5b-af1c-4a3d-a815-d74135ccaae9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:49.966445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:49.967187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:49.967410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:49.967681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:49.968583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=1d6906ac8c4644eeb5bc8ca0a350e99c) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:49.968896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:49.972674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:49.986082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 75/152] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:49 2022] GET /identity/v3/role_assignments?user.id=1d6906ac8c4644eeb5bc8ca0a350e99c&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:49.988345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd3585-97c3-415c-bf36-3648f0fc98bc None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:49.990762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.035181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.036002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.036288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.036559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.048805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:50.055065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:50.058319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:50.059239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:50.060185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 78/153] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:50.067714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.119212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:50.176434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb647a-b57a-491f-a52e-7aab050a3d85 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:50.178041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a50a6-b193-446c-8bf5-2704a06691e6 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:50.215798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.217615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.217615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.217615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.217829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:50.218074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:50.219490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:50.229439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 79/154] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:50.235358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.256586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.257253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.257589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.257864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.365047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68a87c-3894-4e80-b9d1-a677483b776e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:50.367810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:50.390030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:50.392093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:50.402477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.403646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.404135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.404621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.406148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:50.407818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:50.410269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 80/155] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:50 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:50.416185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.465393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:50.528594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09be40-cca5-40af-91dd-90cb9c020872 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:50.585973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 76/156] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:50.587942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462a3a9-85c5-468f-bd75-9827701d83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.588814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462a3a9-85c5-468f-bd75-9827701d83ec None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.589309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462a3a9-85c5-468f-bd75-9827701d83ec None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.589773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462a3a9-85c5-468f-bd75-9827701d83ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.591602 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.677701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:50.745859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3290-db2a-4117-9b78-7d24eb589dcc None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:50.775808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462a3a9-85c5-468f-bd75-9827701d83ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:50.778319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462a3a9-85c5-468f-bd75-9827701d83ec None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:50.799041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0e2b5-8e27-460d-a5fa-e5d2d5f4904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.800397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0e2b5-8e27-460d-a5fa-e5d2d5f4904e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.800687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0e2b5-8e27-460d-a5fa-e5d2d5f4904e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.800968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0e2b5-8e27-460d-a5fa-e5d2d5f4904e None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.802561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0e2b5-8e27-460d-a5fa-e5d2d5f4904e None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:50.804057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0e2b5-8e27-460d-a5fa-e5d2d5f4904e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:50.805934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 77/157] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:50 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:50.811881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717b21d-e247-4809-ab98-c9c7d02563b1 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.812143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717b21d-e247-4809-ab98-c9c7d02563b1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.812391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717b21d-e247-4809-ab98-c9c7d02563b1 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.813878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 78/158] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:50.817408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717b21d-e247-4809-ab98-c9c7d02563b1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.821325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:50.895415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 81/159] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:50.897012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89889-b8e8-4e3b-a87a-674d305b6bdd None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.897301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89889-b8e8-4e3b-a87a-674d305b6bdd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.897576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89889-b8e8-4e3b-a87a-674d305b6bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.914309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e9f89889-b8e8-4e3b-a87a-674d305b6bdd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:50.915500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e9f89889-b8e8-4e3b-a87a-674d305b6bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:50.930754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717b21d-e247-4809-ab98-c9c7d02563b1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:50.967062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:50.967722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.968006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.968282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.969154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=05d0278bf67f4654af88f4f7c25092c2) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:50.969431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:50.970899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff419997-ee16-47c3-a31e-d549cfc282b9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:50.978490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 79/160] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:50 2022] GET /identity/v3/role_assignments?user.id=05d0278bf67f4654af88f4f7c25092c2&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:50.979606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d87dc8-6397-4040-b0af-0f72815c0eea None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:50.979888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d87dc8-6397-4040-b0af-0f72815c0eea None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:50.980161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d87dc8-6397-4040-b0af-0f72815c0eea None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:50.982965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 80/161] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:50.982965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d87dc8-6397-4040-b0af-0f72815c0eea None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:50.997985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 82/162] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:51.000231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f05fa-a115-4a4a-880a-6731781372c7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.000526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f05fa-a115-4a4a-880a-6731781372c7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.000807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f05fa-a115-4a4a-880a-6731781372c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.006337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.019727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-149f05fa-a115-4a4a-880a-6731781372c7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:51.020927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-149f05fa-a115-4a4a-880a-6731781372c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:51.068069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.098367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 83/163] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:51.100261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cce4e6-6b34-4f74-84bf-18a4ea2985e0 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.100555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cce4e6-6b34-4f74-84bf-18a4ea2985e0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.100835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cce4e6-6b34-4f74-84bf-18a4ea2985e0 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.106684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 84/164] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:51.106684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cce4e6-6b34-4f74-84bf-18a4ea2985e0 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.119144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d87dc8-6397-4040-b0af-0f72815c0eea None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.156956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cb19c-b389-466f-9b15-23b34c2da268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.157834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cb19c-b389-466f-9b15-23b34c2da268 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.158113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cb19c-b389-466f-9b15-23b34c2da268 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.158387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cb19c-b389-466f-9b15-23b34c2da268 None admin] PATH_INFO: `/v3/projects/02b54d1a76e746ee96a3752099697e08` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.160185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cb19c-b389-466f-9b15-23b34c2da268 None admin] RBAC: Authorizing `identity:get_project(project_id=02b54d1a76e746ee96a3752099697e08)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:51.161649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cb19c-b389-466f-9b15-23b34c2da268 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:51.163433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 81/165] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:50 2022] GET /identity/v3/projects/02b54d1a76e746ee96a3752099697e08 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:51.164673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.164957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.165228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.184725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:51.185971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:51.249052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cce4e6-6b34-4f74-84bf-18a4ea2985e0 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.274854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 82/166] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:51.274854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.295047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.301046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.301845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.302130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.302406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.303598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:51.305066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:51.306881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 85/167] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:51.307547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.334743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.372703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cea313-8d5e-4cd0-9e8b-ae697c7f3623 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.405867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.406387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a8022-40e3-40a6-a40d-c65f2809ddce None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.406654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.406967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.417147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:51.425503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:51.426869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:51.427852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 83/168] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:51.432051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.443201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.443852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.444137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.444411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.445241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=02b54d1a76e746ee96a3752099697e08, user.id=1d6906ac8c4644eeb5bc8ca0a350e99c) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:51.445553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:51.447017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:51.453081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 86/169] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/role_assignments?user.id=1d6906ac8c4644eeb5bc8ca0a350e99c&scope.project.id=02b54d1a76e746ee96a3752099697e08&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:51.453677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.524194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.552928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263a476-4849-4e55-ac5e-159bb2a4bc50 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.570364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a62d1-36ae-46d0-ac42-d70a896d9c9a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.581183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.594918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.595797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.596086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.596364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.609177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:51.614705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.615421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.615713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.615991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.616721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:51.618213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:51.618396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:51.619486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:51.620308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 84/170] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:51.620962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.647079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 87/171] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:03:51 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:51.647079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.772002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ee95-efce-4584-af49-4343b41bade6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.787986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564237c0-27fa-46e3-af14-4460875e476b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.810357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.811080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.811390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.812136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.812545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.813506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:51.813970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:51.815861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:51.829497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 85/172] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:51.830753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.833453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:51.834404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:51.835657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:51.835942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:51.836725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:51.837036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:51.838468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:51.849664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 88/173] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:51.855811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:51.886777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:51.957449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b6a11-fc1b-4b06-82df-64af884e9399 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:51.985189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6588c2-307c-47ec-a777-9f4772d57976 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:52.005396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.006508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.006793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.007099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] PATH_INFO: `/v3/users/728ba87f8e3a45838c2f69e43ee6d0ad` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.008346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] RBAC: Authorizing `identity:get_user(user_id=728ba87f8e3a45838c2f69e43ee6d0ad)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.010213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.012110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 86/174] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/users/728ba87f8e3a45838c2f69e43ee6d0ad => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:52.017614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:52.027677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.028570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.028960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.029354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] PATH_INFO: `/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.030667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] RBAC: Authorizing `identity:get_user(user_id=1d6906ac8c4644eeb5bc8ca0a350e99c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.032299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.034930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 89/175] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:51 2022] GET /identity/v3/users/1d6906ac8c4644eeb5bc8ca0a350e99c => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:52.043116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:52.097621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:52.147940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43449e8-0d00-4eb5-8284-702b8bd640fd None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:52.163098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af19149-9e88-468d-8cfc-2b53e1a2c47c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:52.167082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:52.195983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.197125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.197508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.197962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.199363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.201119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.203005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 87/176] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:52 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:52.205655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.206862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.207154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.207428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.209878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:52.279637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.281127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.337036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:52.338207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072468f-8c13-4cc2-9371-0bc484fe11d1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:52.377582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.378507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.378910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.379186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/728ba87f8e3a45838c2f69e43ee6d0ad/roles/f6ce35b028e548f68db97d1f5bfc955e` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.381472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] RBAC: Authorizing `identity:create_grant(project_id=33a909cd233e4d5abcb4773bc2a46e27, user_id=728ba87f8e3a45838c2f69e43ee6d0ad, role_id=f6ce35b028e548f68db97d1f5bfc955e)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.383091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.398945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 88/177] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:03:52 2022] PUT /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/users/728ba87f8e3a45838c2f69e43ee6d0ad/roles/f6ce35b028e548f68db97d1f5bfc955e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:03:52.401313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 90/178] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:03:52.406995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:52.461549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:52.555494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01382838-f61d-417b-b6f4-7cef2d540bab None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:52.598539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f39ef-e827-457c-8be1-6f45a5755901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.599489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f39ef-e827-457c-8be1-6f45a5755901 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.599815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f39ef-e827-457c-8be1-6f45a5755901 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.600089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f39ef-e827-457c-8be1-6f45a5755901 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.601333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f39ef-e827-457c-8be1-6f45a5755901 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.602854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f39ef-e827-457c-8be1-6f45a5755901 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.604949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 89/179] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:52 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:52.610561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:52.745681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8989c-4abd-47b3-b3d9-3958a99c73ee None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:52.783009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:52.783669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:52.783950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:52.784751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:52.785589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] RBAC: Adding query filter params (role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=1d6906ac8c4644eeb5bc8ca0a350e99c) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:52.785903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:52.787392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a45df3-47b4-4e41-a458-1150c48e8b58 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:52.795673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 91/180] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:52 2022] GET /identity/v3/role_assignments?user.id=1d6906ac8c4644eeb5bc8ca0a350e99c&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:54.200018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5851-5c9c-4dd7-81cf-ee37a710bdb3 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.200562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5851-5c9c-4dd7-81cf-ee37a710bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.200562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5851-5c9c-4dd7-81cf-ee37a710bdb3 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.202069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 90/181] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:54.208214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11d7d2-ae50-49c8-b2ef-b8238a8f488a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.208214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11d7d2-ae50-49c8-b2ef-b8238a8f488a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.208214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11d7d2-ae50-49c8-b2ef-b8238a8f488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.226184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bf11d7d2-ae50-49c8-b2ef-b8238a8f488a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:54.227722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bf11d7d2-ae50-49c8-b2ef-b8238a8f488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:54.237852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:54.306966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 92/182] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:54.331427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e5851-5c9c-4dd7-81cf-ee37a710bdb3 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:54.449339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1dbb5-8e31-40ba-a636-e6941fb40cfb None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.449630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1dbb5-8e31-40ba-a636-e6941fb40cfb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.449900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1dbb5-8e31-40ba-a636-e6941fb40cfb None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.451769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 93/183] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:54.451874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e5851-5c9c-4dd7-81cf-ee37a710bdb3 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:54.456781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.457070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.457348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.462966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:54.474105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:54.475324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:54.487779 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:54.488584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.488866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.489140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.499857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:54.508069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:54.508945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:54.509731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 91/184] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:54.515753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:54.547090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 94/185] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:54.571009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:54.621205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a8a94-e8f4-4ef1-b574-8fb573e7ebef None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:54.655956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:54.659695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:54.660362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.660642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.660920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.661685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:54.661987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:54.663420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a78e7c-45ae-450a-aeb0-5347816edf81 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:54.672721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 92/186] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:54.673832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21be81c-4e53-4a75-9997-41dfbabb649f None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.674120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21be81c-4e53-4a75-9997-41dfbabb649f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.674395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21be81c-4e53-4a75-9997-41dfbabb649f None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.676180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 93/187] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:54.679956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21be81c-4e53-4a75-9997-41dfbabb649f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:54.688273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aadf9d-7ff2-4bdf-ae69-b280cbb09f3c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:54.711968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:54.729150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21da4b1-b896-408c-8c63-d03995329a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:54.729959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da4b1-b896-408c-8c63-d03995329a28 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.730240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da4b1-b896-408c-8c63-d03995329a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.730519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da4b1-b896-408c-8c63-d03995329a28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.741904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da4b1-b896-408c-8c63-d03995329a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-b21da4b1-b896-408c-8c63-d03995329a28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:54.750464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:54.751459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:54.752250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 95/188] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:03:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:54.752959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.758452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.758452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.774972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:54.776142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:54.793396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21be81c-4e53-4a75-9997-41dfbabb649f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:54.828098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:54.828977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.829292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.829525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] PATH_INFO: `/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:54.843299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] RBAC: Authorizing `identity:get_user(user_id=dc1aa9d1b37d4e7cae7a876b3ca56637)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:54.844886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:54.851271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 94/189] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:54 2022] GET /identity/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:54.851271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:54.857229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 96/190] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:54.862871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:54.932517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:54.953421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:54.963545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf71f-755d-4d7a-a76e-fb246c0e4fc2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:54.975389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc137-a92e-438e-8660-b3049e6ab657 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:54.998496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:54.999265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:54.999546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:54.999820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.000562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:55.000872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.002273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.008786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.009429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.009709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.009953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 95/191] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:03:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:55.010040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.010712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.078165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.079644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.113856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3d598-3aa4-4b12-9f27-85358afae2e2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.137803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.147688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b2632-1d5d-4f17-ba5b-26a2fa6de73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.148469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b2632-1d5d-4f17-ba5b-26a2fa6de73b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.148752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b2632-1d5d-4f17-ba5b-26a2fa6de73b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.149027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b2632-1d5d-4f17-ba5b-26a2fa6de73b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.150193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b2632-1d5d-4f17-ba5b-26a2fa6de73b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.151647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b2632-1d5d-4f17-ba5b-26a2fa6de73b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.153250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 96/192] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:55.154400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a8025-9d76-4f2b-b325-b25401537df5 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.154684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a8025-9d76-4f2b-b325-b25401537df5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.154988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a8025-9d76-4f2b-b325-b25401537df5 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.158064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 97/193] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:55.158064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a8025-9d76-4f2b-b325-b25401537df5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.180601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.181592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 97/194] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:55.182498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.263348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a8025-9d76-4f2b-b325-b25401537df5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.282918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e565fb-6796-40d6-88d2-58da3c45cebb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.296761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377fb72-8211-4af6-a629-311e0695e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.297655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377fb72-8211-4af6-a629-311e0695e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.297937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377fb72-8211-4af6-a629-311e0695e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.298210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377fb72-8211-4af6-a629-311e0695e6b4 None admin] PATH_INFO: `/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.299422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377fb72-8211-4af6-a629-311e0695e6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=dc1aa9d1b37d4e7cae7a876b3ca56637)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.301721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377fb72-8211-4af6-a629-311e0695e6b4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.303369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 98/195] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:55.304496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.304787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.305117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.321827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:55.322662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.323031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:55.323352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.323633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.323906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.324600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.326032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.345235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 98/196] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:03:55 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:03:55.346142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.371495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.413047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 99/197] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:55.413868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.436716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.479599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a78b9-d8f5-4438-b899-4616d9c834d2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.519423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.520216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.520543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.520774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.522395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.524034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.525906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 99/198] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:55.531358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f2dc-ca17-4775-add2-d79fd114d081 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.531528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.572912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.573851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.574305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.574782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.640551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.652822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.653804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568f196-682a-4344-a85a-ab9e695a0566 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.654305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.676818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.697815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.698621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.698946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.699208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.711560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:55.720349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:55.721230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:55.722571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 100/199] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:55.723328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.770450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 100/200] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:55.771670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.841914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178a9a2-c2c9-4c74-8c35-d6dd8a139cc5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.882513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.883746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.884567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.885320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.886610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=dc1aa9d1b37d4e7cae7a876b3ca56637) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:55.887541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.889639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.894724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f9177-28ef-45bc-ba24-91b92e7c3f12 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:55.898692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 101/201] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/role_assignments?user.id=dc1aa9d1b37d4e7cae7a876b3ca56637&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:55.899765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:55.915398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.922608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:55.936271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:55.937627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:55.938214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:55.938795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:55.940896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:55.941301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:55.943023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:55.952778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 101/202] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:55.959730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:56.042734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d86a8-a144-4469-aa66-6e0518c32b95 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:56.089134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:56.090417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:56.090803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:56.091206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:56.092600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:56.094092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca767eb-263c-421f-a66c-a27b723244ff None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:56.094267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:56.096520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 102/203] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:56.102857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:56.152438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033cd55-5123-4725-976f-a92192358419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:56.154254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033cd55-5123-4725-976f-a92192358419 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:56.154763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033cd55-5123-4725-976f-a92192358419 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:56.155517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033cd55-5123-4725-976f-a92192358419 None admin] PATH_INFO: `/v3/users/728ba87f8e3a45838c2f69e43ee6d0ad` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:56.160994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033cd55-5123-4725-976f-a92192358419 None admin] RBAC: Authorizing `identity:get_user(user_id=728ba87f8e3a45838c2f69e43ee6d0ad)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:56.166495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033cd55-5123-4725-976f-a92192358419 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:56.169746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 102/204] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:55 2022] GET /identity/v3/users/728ba87f8e3a45838c2f69e43ee6d0ad => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:56.176686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033cd55-5123-4725-976f-a92192358419 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:56.233655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:56.265340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:56.270755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d28ee-304f-4020-b555-872170c2ba65 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:56.328837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033cd55-5123-4725-976f-a92192358419 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:56.348461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:56.349128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:56.349409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:56.349684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:56.350526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] RBAC: Adding query filter params (role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=dc1aa9d1b37d4e7cae7a876b3ca56637) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:56.350855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:56.352302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:56.359894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 103/205] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:56 2022] GET /identity/v3/role_assignments?user.id=dc1aa9d1b37d4e7cae7a876b3ca56637&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:03:56.374612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:56.375715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:56.376227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:56.376710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:56.495208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:56.497067 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:56.571885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:56.634587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 103/206] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:03:56.640315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:56.790908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37850db3-db62-41a1-87c1-42a24467792c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:56.830616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279910bb-0eab-40e3-b289-d29cd6be02f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:56.832401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279910bb-0eab-40e3-b289-d29cd6be02f3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:56.832911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279910bb-0eab-40e3-b289-d29cd6be02f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:56.833384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279910bb-0eab-40e3-b289-d29cd6be02f3 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:56.834789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279910bb-0eab-40e3-b289-d29cd6be02f3 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:56.836483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279910bb-0eab-40e3-b289-d29cd6be02f3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:56.838811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 104/207] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:56 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:56.848331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:56.970345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab236-30c8-48b4-a266-89b31a7666e1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:57.010368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:57.011059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:57.011345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:57.011622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:57.012468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] RBAC: Adding query filter params (role.id=f6ce35b028e548f68db97d1f5bfc955e, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27, user.id=728ba87f8e3a45838c2f69e43ee6d0ad) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:57.012784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:57.014225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5977f8f3-c838-460b-8253-154d289fa2a3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:57.021208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 104/208] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:56 2022] GET /identity/v3/role_assignments?user.id=728ba87f8e3a45838c2f69e43ee6d0ad&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=f6ce35b028e548f68db97d1f5bfc955e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:58.034034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4464-bf67-4680-8378-bfe87307051d None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.035621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4464-bf67-4680-8378-bfe87307051d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.036108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d4464-bf67-4680-8378-bfe87307051d None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.037690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 105/209] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:58.043146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.044056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.044731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.064197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:58.066027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:58.074136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.088866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ed6e-04e3-4dc1-9e38-c9d9208e511e None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.088866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ed6e-04e3-4dc1-9e38-c9d9208e511e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.088866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ed6e-04e3-4dc1-9e38-c9d9208e511e None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.088866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 106/210] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:58.090896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.091188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.091470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.108760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:58.109947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:58.118120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.141387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 105/211] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:03:58.166866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:58.184952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 107/212] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:58.205488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:58.272854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dde77b-606d-44d0-bb85-38f27ffd221b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:58.311765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:58.312877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.313164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.313443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.314923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:58.317005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:58.322907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 106/213] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:58.325275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:58.340079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.348321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276d275-b627-4168-ad84-41e00f4625ff None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:58.392131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:58.393418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.394491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.394777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.410195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:58.419743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:58.420695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:58.421481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 108/214] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:03:58.432750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:58.435790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.466654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a70d-bd70-46d6-bb2b-68565292fa7f None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:58.505013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:58.505685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.505966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.506242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.506995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:58.508437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:58.531555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 107/215] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:03:58 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:03:58.552247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065a302-b528-46e5-aa87-ace0ebf8859e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:58.593905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:58.595069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.595612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.596126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.597224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:58.597743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:58.599557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a2e63-ee27-4f92-84e8-1cd8d07a39f3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:58.610109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 109/216] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:03:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:58.616267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:58.692384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.735299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489369ea-fbe3-4564-80e6-58e58d966b3e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:58.771745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:58.772625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.772904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.773175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] PATH_INFO: `/v3/users/2a5350040a4e435c8f1e530bb39ff789` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.775919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d8f1b-02af-40d5-9958-379938867bf8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.776427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d8f1b-02af-40d5-9958-379938867bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.776913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d8f1b-02af-40d5-9958-379938867bf8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.778479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 110/217] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:03:58.784022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3521e7-1e04-4c60-894a-832dd3b8d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.784597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3521e7-1e04-4c60-894a-832dd3b8d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.785087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3521e7-1e04-4c60-894a-832dd3b8d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.787592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] RBAC: Authorizing `identity:get_user(user_id=2a5350040a4e435c8f1e530bb39ff789)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:58.789086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:58.790811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 108/218] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:58 2022] GET /identity/v3/users/2a5350040a4e435c8f1e530bb39ff789 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:58.796766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:58.804687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fb3521e7-1e04-4c60-894a-832dd3b8d0e8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:58.806251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fb3521e7-1e04-4c60-894a-832dd3b8d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:58.883122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.890153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 111/219] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:58.896347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea28b3-e859-41e9-a8d9-fa3aa3720530 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.896651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea28b3-e859-41e9-a8d9-fa3aa3720530 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.897190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea28b3-e859-41e9-a8d9-fa3aa3720530 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.898856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 112/220] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:03:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:03:58.904155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.904512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.904859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:58.920664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed09d24-1a72-4293-abd6-18fb429fdc6a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:58.923031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:03:58.924297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:03:58.950102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:58.968092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:58.969098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:58.969631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:58.970325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.014035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 113/221] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:03:59.014689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.062578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:59.065036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:59.139276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d370e7-c023-43ae-a169-7ffff6679734 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.169152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:59.172211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 109/222] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:03:59.178068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.183915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.184830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.185278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.185671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.198802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:59.207143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:59.208143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:59.209012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:03:59.209805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:03:59.214571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 114/223] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:59.215137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.285106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0d08f-0112-4950-b446-2ee6825be8d3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.320738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.321551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.321832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.322108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.333628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c9aad-638a-4d54-99ee-a7471454674c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.334263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:03:59.342952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:03:59.343833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:03:59.344621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 110/224] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:03:59.345290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.387546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.389156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.389503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.389728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.390969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:59.393650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:59.395608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 115/225] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:59.396152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.460932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:59.467420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:59.483488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866307be-44d4-4a37-983d-4e15cb1a3230 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.524849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.526299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.526727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.527174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.528243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:59.528991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:59.530972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:59.535672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068b43-5ccc-46db-ad78-aa9a850de07e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.541963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 111/226] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:59.543007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.576538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.577218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.577644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.577928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.578846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:59.579134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:59.580570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:59.590201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 116/227] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:59.591184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.668180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c9270-d353-476e-8aea-bba23f047528 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.721318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1033-b674-4e59-89ef-cbae1e10a68b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.740086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:59.744138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.744907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.745199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.745436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.746273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=2a5350040a4e435c8f1e530bb39ff789) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:03:59.746582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:59.748064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:59.754764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 112/228] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/role_assignments?user.id=2a5350040a4e435c8f1e530bb39ff789&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:59.755782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.769627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:03:59.776585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:03:59.777483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:03:59.777764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:03:59.778034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] PATH_INFO: `/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:03:59.779616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] RBAC: Authorizing `identity:get_user(user_id=dc1aa9d1b37d4e7cae7a876b3ca56637)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:03:59.781457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:03:59.788716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 117/229] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:03:59.795733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:03:59.963089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9d00b-955e-4fb8-9e5c-fc695b65cb70 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:03:59.965903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e879d-3114-482b-99f1-6f2b1cf8f416 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:00.019995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:00.020893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:00.021182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:00.021458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:00.027827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:00.029345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:00.031237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 118/230] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:00.041740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:00.069590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:00.071317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:00.071817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:00.072090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] PATH_INFO: `/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:00.074240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] RBAC: Authorizing `identity:get_user(user_id=dc1aa9d1b37d4e7cae7a876b3ca56637)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:00.087225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:00.089867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 113/231] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:03:59 2022] GET /identity/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637 => generated 283 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:00.096350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:00.156689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:00.272475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4d64f-0ff1-4bc1-a469-efdb8320d53b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:00.323421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:00.350019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062d856-89d1-4f5d-ad57-be4c9e8d56e4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:00.386088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:00.387003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:00.387291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:00.387576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:00.388749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:00.390212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:00.391958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 114/232] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:00 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:00.397526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:00.398428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c568496-e445-4f5c-aa79-0f15467406da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:00.399582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568496-e445-4f5c-aa79-0f15467406da None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:00.400066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568496-e445-4f5c-aa79-0f15467406da None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:00.400539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c568496-e445-4f5c-aa79-0f15467406da None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b/users/dc1aa9d1b37d4e7cae7a876b3ca56637/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:00.411686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c568496-e445-4f5c-aa79-0f15467406da None admin] RBAC: Authorizing `identity:create_grant(project_id=633f8860113345a89d56f3146f465a5b, user_id=dc1aa9d1b37d4e7cae7a876b3ca56637, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:00.413447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c568496-e445-4f5c-aa79-0f15467406da None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:00.442331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 119/233] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:00 2022] PUT /identity/v3/projects/633f8860113345a89d56f3146f465a5b/users/dc1aa9d1b37d4e7cae7a876b3ca56637/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:00.508608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72707ea-f51b-4a6f-9163-d593ee7db2d7 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:00.545197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45187e-0e2e-4a5d-bf01-23773f672fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:00.546505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45187e-0e2e-4a5d-bf01-23773f672fde None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:00.547046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45187e-0e2e-4a5d-bf01-23773f672fde None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:00.547542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45187e-0e2e-4a5d-bf01-23773f672fde None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b/users/dc1aa9d1b37d4e7cae7a876b3ca56637/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:00.550666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45187e-0e2e-4a5d-bf01-23773f672fde None admin] RBAC: Authorizing `identity:create_grant(project_id=633f8860113345a89d56f3146f465a5b, user_id=dc1aa9d1b37d4e7cae7a876b3ca56637, role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:00.552408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45187e-0e2e-4a5d-bf01-23773f672fde None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:00.569090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 115/234] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:00 2022] PUT /identity/v3/projects/633f8860113345a89d56f3146f465a5b/users/dc1aa9d1b37d4e7cae7a876b3ca56637/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:04:01.271390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0466b-a72a-4eb4-b775-3598a4ea8e62 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.272260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0466b-a72a-4eb4-b775-3598a4ea8e62 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.272545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0466b-a72a-4eb4-b775-3598a4ea8e62 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.274088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 120/235] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:01.279803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cd432-21b3-4895-9eef-10d3289bb0da None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.280239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cd432-21b3-4895-9eef-10d3289bb0da None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.280434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cd432-21b3-4895-9eef-10d3289bb0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.292714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:01.305320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-fe2cd432-21b3-4895-9eef-10d3289bb0da None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:01.306581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fe2cd432-21b3-4895-9eef-10d3289bb0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:01.389461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 116/236] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:01.433393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0466b-a72a-4eb4-b775-3598a4ea8e62 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:01.434132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874a672-922e-46ca-bde0-0e03234728ce None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.434422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874a672-922e-46ca-bde0-0e03234728ce None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.434700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874a672-922e-46ca-bde0-0e03234728ce None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.436391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 117/237] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:01.442266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.442551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.442844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.472543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:01.474249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:01.595447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 118/238] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:01.607554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0466b-a72a-4eb4-b775-3598a4ea8e62 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:01.619225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:01.643198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:01.676146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:01.676952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.677244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.677524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.681076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:01.682551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:01.684229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 121/239] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:01.691038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:01.731221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:01.804274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32336db2-3794-4d2a-8b73-aceddd30988b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:01.874429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:01.875169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.875457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.875731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.897285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cff0e7-d1b2-47d4-975f-0ba1fd7f394f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:01.898788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:01.917706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:01.918643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:01.919485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:01.921714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 119/240] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:01.924541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:01.972787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:01.973476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:01.973744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:01.974020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:01.974732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:01.976216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:02.021767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 122/241] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:01 2022] POST /identity/v3/users => generated 293 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:02.069074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba730f8-7591-4b4b-8022-63a4289a176e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:02.098975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:02.111521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:02.112501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.113063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.113341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.114137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:02.114443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:02.115925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:02.131208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 120/242] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:02.137123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:02.210743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:02.303590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bec8d-07a3-4a9f-a554-e2db695a4681 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:02.339212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62799d9-03e1-4cbf-80da-a2476bf9f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:02.340092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62799d9-03e1-4cbf-80da-a2476bf9f83d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.340379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62799d9-03e1-4cbf-80da-a2476bf9f83d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.340656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62799d9-03e1-4cbf-80da-a2476bf9f83d None admin] PATH_INFO: `/v3/users/157dfaa196bc42d5962961d24c698eac` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.354358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62799d9-03e1-4cbf-80da-a2476bf9f83d None admin] RBAC: Authorizing `identity:get_user(user_id=157dfaa196bc42d5962961d24c698eac)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:02.355841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62799d9-03e1-4cbf-80da-a2476bf9f83d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:02.357841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 123/243] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:02 2022] GET /identity/v3/users/157dfaa196bc42d5962961d24c698eac => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:02.363179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:02.480738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d519fc-e415-477e-a517-514d0ba83fba None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:02.511390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71064-8a28-4598-b396-306201d3a5a9 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.511684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71064-8a28-4598-b396-306201d3a5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.512037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a71064-8a28-4598-b396-306201d3a5a9 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.513665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 124/244] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:02.520008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:02.520833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.521124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.521385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.521486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.521948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.522433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.552839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:02.554057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:02.585866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:02.598989 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:02.600991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:02.644397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 125/245] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:02.668602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:02.701687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:02.730864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 121/246] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:02.736153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:02.797626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4968b8-b0b2-4ce9-972f-275736a375ea None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:02.851649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:02.852473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.852758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.853031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.861607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:02.869316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:02.877323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:02.878863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:02.879680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:02.881219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680aff9-6d93-4a3b-9823-5458ff6dfcd0 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:02.881365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 126/247] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:02.885836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:02.924230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:02.925648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:02.926085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:02.926363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:02.928500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:02.929992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:02.934911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 122/248] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:02 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:02.938333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:02.983131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:03.004709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c524128-ae6c-4bfb-b39a-6eaf8737ddce None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:03.040284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:03.041048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.041404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.041754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.042587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:03.043009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:03.044536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:03.054338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 127/249] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:03.060049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:03.063344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bef4b-a5cc-4bfb-b762-a3bc203f76f5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:03.099406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:03.100093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.100368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.100640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.101487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] RBAC: Adding query filter params (role.id=f6ce35b028e548f68db97d1f5bfc955e, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=157dfaa196bc42d5962961d24c698eac) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:03.101795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:03.103265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:03.114686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 123/250] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:02 2022] GET /identity/v3/role_assignments?user.id=157dfaa196bc42d5962961d24c698eac&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=f6ce35b028e548f68db97d1f5bfc955e => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:03.177934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f75bb8-0ce6-458b-abf2-746534dca622 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:03.223744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:03.224676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.224967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.225246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] PATH_INFO: `/v3/users/2a5350040a4e435c8f1e530bb39ff789` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.226855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5350040a4e435c8f1e530bb39ff789)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:03.228379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:03.230467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 128/251] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:03 2022] GET /identity/v3/users/2a5350040a4e435c8f1e530bb39ff789 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:03.236254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:03.274130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:03.363338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4575bf-2eb9-4e3d-840b-ad791cd90301 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:03.400509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f97c2d-d789-4ed3-aee1-ecc6bc58033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:03.401382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97c2d-d789-4ed3-aee1-ecc6bc58033f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.401658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97c2d-d789-4ed3-aee1-ecc6bc58033f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.401928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97c2d-d789-4ed3-aee1-ecc6bc58033f None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.403151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f97c2d-d789-4ed3-aee1-ecc6bc58033f None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:03.404609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f97c2d-d789-4ed3-aee1-ecc6bc58033f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:03.406350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 124/252] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:03 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:03.412769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:03.443468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:03.496044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8d1e8-52fc-430e-b938-76ea2dd3919f None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.496664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8d1e8-52fc-430e-b938-76ea2dd3919f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.497136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8d1e8-52fc-430e-b938-76ea2dd3919f None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.499785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 125/253] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:03.503714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b88d63-5719-4937-8526-8b1d30769ebe None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.504226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b88d63-5719-4937-8526-8b1d30769ebe None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.504714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b88d63-5719-4937-8526-8b1d30769ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.531533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-98b88d63-5719-4937-8526-8b1d30769ebe None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:03.533062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-98b88d63-5719-4937-8526-8b1d30769ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:03.546870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bc704-cdb7-4dcd-94af-f97d0d9263a8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:03.576062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:03.577034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.577238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.577680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b/users/2a5350040a4e435c8f1e530bb39ff789/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.580062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=633f8860113345a89d56f3146f465a5b, user_id=2a5350040a4e435c8f1e530bb39ff789, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:03.582383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:03.620691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 129/254] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:03 2022] PUT /identity/v3/projects/633f8860113345a89d56f3146f465a5b/users/2a5350040a4e435c8f1e530bb39ff789/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:03.653476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 126/255] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:03.693036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:03.761734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7adfec-a689-463f-b8d6-1b4a240ce772 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.762325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7adfec-a689-463f-b8d6-1b4a240ce772 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.762812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7adfec-a689-463f-b8d6-1b4a240ce772 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.765858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 127/256] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:03.779512 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.780126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.780593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.784106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:03.817160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:03.818728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:03.819373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7fe41-39bf-4b45-93ad-8659ff30b7d4 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:03.841904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:03.864500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:03.866048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:03.866513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:03.867735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:03.892145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:03.909324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:03.910478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:03.911497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:03.914845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 130/257] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:03.920881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:03.968062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 128/258] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:03.999935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:04.083299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e61bb2-49e9-4180-b2aa-bdbf3d6532cb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:04.109625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140c44a-9bc2-4117-9949-c3d5d71d9737 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:04.135514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:04.147686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:04.148571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:04.149293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:04.149720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:04.150862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:04.151332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:04.153358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:04.178084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 131/259] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:04.183657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:04.187543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:04.188366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:04.190005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:04.190363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:04.224513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:04.233491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:04.234394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:04.235246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:04.240617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 129/260] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:04.246311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:04.266663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:04.414914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29208e49-47f2-42e1-a7e4-ebd9c714904a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:04.469854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c969cf-fe49-42d7-a1b8-294e306f05ef None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:04.514004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:04.514923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:04.515208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:04.515482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] PATH_INFO: `/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:04.519531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] RBAC: Authorizing `identity:get_user(user_id=dc1aa9d1b37d4e7cae7a876b3ca56637)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:04.520997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:04.522849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 132/261] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:04 2022] GET /identity/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637 => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:04.528806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:04.536567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:04.537698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:04.539043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:04.539554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:04.540510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:04.541014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:04.542596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:04.558303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 130/262] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:04.563554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:04.673393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:04.690343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:04.755956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff09d3d-271b-4a75-b32a-713a8bddcf02 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:04.760857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa618c84-0ef3-4420-9062-29cf24a17932 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:04.816900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:04.817554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:04.817837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:04.818107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:04.837797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:04.838856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:04.839251 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:04.839624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] PATH_INFO: `/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:04.840934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] RBAC: Authorizing `identity:get_user(user_id=dc1aa9d1b37d4e7cae7a876b3ca56637)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:04.842478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:04.844563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 131/263] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:04 2022] GET /identity/v3/users/dc1aa9d1b37d4e7cae7a876b3ca56637 => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:04.851849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:04.938204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:04.947696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:05.012550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe7b32-15b9-411c-9dcd-6e575fe808ea None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:05.059363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:05.075507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:05.076455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.076893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.077270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.100855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:05.189198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:05.191461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:05.193352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 133/264] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:05.203882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:05.338754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 132/265] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:05.345327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:05.421871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:05.519015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55fc6-bffc-4ba0-b94e-24f851b6db71 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:05.544884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db314cb-011e-4a5a-a2b6-b941ee5e649d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:05.567314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:05.568180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.568462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.568737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.569929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:05.571418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:05.579292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 133/266] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:05 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:05.585612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:05.660788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:05.661676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.661951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.662222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.667612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:05.669076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:05.670844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 134/267] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:05 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:05.676465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:05.752274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:05.802145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8959a-00e7-4b10-8d0f-2269eb65bf60 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:05.811782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:05.838277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:05.838967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.839264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.839546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.840449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] RBAC: Adding query filter params (role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=dc1aa9d1b37d4e7cae7a876b3ca56637) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:05.840701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:05.842149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2941-adc8-4799-9eeb-cf52d2ee7c90 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:05.857223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 134/268] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:05 2022] GET /identity/v3/role_assignments?user.id=dc1aa9d1b37d4e7cae7a876b3ca56637&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:05.858673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b32cb-72bb-406e-a935-1998f935b516 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.859086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b32cb-72bb-406e-a935-1998f935b516 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.859369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b32cb-72bb-406e-a935-1998f935b516 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.860762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 135/269] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:05.865097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235a1a6-0dfb-439e-9165-b2d53daf4f51 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:05.869213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc377d3-8ee0-4dd1-ba28-cf94ef5b7bf6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.869500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc377d3-8ee0-4dd1-ba28-cf94ef5b7bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.869784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc377d3-8ee0-4dd1-ba28-cf94ef5b7bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.900250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-bbc377d3-8ee0-4dd1-ba28-cf94ef5b7bf6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:05.901622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bbc377d3-8ee0-4dd1-ba28-cf94ef5b7bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:05.904013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:05.904687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:05.904967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:05.905241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:05.906085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=dc1aa9d1b37d4e7cae7a876b3ca56637) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:05.906392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:05.907890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226e7dd3-7fe9-4cce-a34a-a6bffbb91579 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:05.915687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 135/270] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:05 2022] GET /identity/v3/role_assignments?user.id=dc1aa9d1b37d4e7cae7a876b3ca56637&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:06.032676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 136/271] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:06.034648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc20e4-84dc-4c0d-abd1-ac19d3773154 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.035095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc20e4-84dc-4c0d-abd1-ac19d3773154 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.035376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc20e4-84dc-4c0d-abd1-ac19d3773154 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.036846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 136/272] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:06.056740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.057031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.057210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.069306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc20e4-84dc-4c0d-abd1-ac19d3773154 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.085606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:06.086809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:06.174448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 137/273] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:06.223691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.237532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:06.251155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc20e4-84dc-4c0d-abd1-ac19d3773154 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:06.262724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:06.287984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:06.288838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.289067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.289427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.290608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:06.292083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:06.298165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 137/274] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:06.299513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.350324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494a3c0-63c8-4fbe-9add-d6ff33431e81 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:06.399890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:06.400698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.401249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.401526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.411982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3586168-9492-4f32-be09-1db90f88f780 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:06.416454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:06.424306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:06.425771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:06.426583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 138/275] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:06.432665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.460405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7ba9b-36ff-4e9d-a1cf-5eb4c30e848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:06.461077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7ba9b-36ff-4e9d-a1cf-5eb4c30e848e None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.461357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7ba9b-36ff-4e9d-a1cf-5eb4c30e848e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.461632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7ba9b-36ff-4e9d-a1cf-5eb4c30e848e None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.462346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7ba9b-36ff-4e9d-a1cf-5eb4c30e848e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:06.463837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7ba9b-36ff-4e9d-a1cf-5eb4c30e848e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:06.488669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 138/276] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:06 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:04:06.532991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91db2d-6ed0-4e85-a585-3821c8f1505d None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.533615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91db2d-6ed0-4e85-a585-3821c8f1505d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.534077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91db2d-6ed0-4e85-a585-3821c8f1505d None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.535795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 139/277] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:06.541572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.542279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.542712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.548293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:06.559027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b750fb8-68dd-4fa7-8ac6-4d3a89ec0e67 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:06.566976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:06.569642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:06.588562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:06.614545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:06.615185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.615500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.616090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.617465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:06.617789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:06.619632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:06.631396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 139/278] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:06.637731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.669247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 140/279] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:06.694479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.777279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66437-20dd-4733-9825-75bb9a289cb2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:06.816659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112db0ba-01ac-4636-89ab-97be50d221a7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:06.821419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:06.823115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.823231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.826713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] PATH_INFO: `/v3/users/071bdd43185844249bbecaf58c78f939` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.837624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] RBAC: Authorizing `identity:get_user(user_id=071bdd43185844249bbecaf58c78f939)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:06.839129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:06.840844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 140/280] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/users/071bdd43185844249bbecaf58c78f939 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:06.847249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.862048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:06.862434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:06.864066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:06.864271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:06.864765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:06.878991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:06.887803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:06.888903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:06.889846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 141/281] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:06.897108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:06.908618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:06.973362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc360596-11c8-4add-8a12-ec32ecdabc53 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.015905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.016580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.017024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.017309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.018672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, user.id=071bdd43185844249bbecaf58c78f939, scope.system=all) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:07.019146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.021234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc446c-ed5c-4ff2-b036-65970d22bdf7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.029011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 141/282] 10.222.0.61 () {56 vars in 1308 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/role_assignments?user.id=071bdd43185844249bbecaf58c78f939&scope.system=all&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:07.029175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba2735-adc6-460f-8649-f0e10478560e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.030323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c818768-7204-4edd-9924-68d6fb591dd3 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.030792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c818768-7204-4edd-9924-68d6fb591dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.031151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c818768-7204-4edd-9924-68d6fb591dd3 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.033342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 142/283] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:07.040805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.041118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.041735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.067894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:07.069207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:07.071389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.072350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.072634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.073890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.074692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:07.075061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.076575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.090747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 142/284] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:07.096077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:07.179297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:07.208725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 143/285] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:07.213909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54357ff-2d0c-4963-9816-159588ffe4d8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.247942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:07.251138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:07.251261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.252850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.253242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.253534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] PATH_INFO: `/v3/users/157dfaa196bc42d5962961d24c698eac` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.254805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] RBAC: Authorizing `identity:get_user(user_id=157dfaa196bc42d5962961d24c698eac)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.256469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.258239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 143/286] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:07 2022] GET /identity/v3/users/157dfaa196bc42d5962961d24c698eac => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:07.273240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:07.436834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488754e4-3c89-4fb9-83b9-4e5936a63371 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.475050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b88e92-0fe2-4ef1-82c9-ce90202e6f1f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.502168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.503513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.503800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.504072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.524234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:07.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:07.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:07.534505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 144/287] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:07.542941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:07.571566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.572447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.572731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.573006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.579553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.581019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.583726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 144/288] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:07 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:07.591706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:07.612254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:07.615235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:07.659662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e1246-d790-4177-878e-60dd1a200615 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.723355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.723942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.724228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.724506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.725266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:07.725572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.729246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.749021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 145/289] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:07.761201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:07.842512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b61366-396a-483a-868c-bb38c3871507 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.909331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.911451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.913407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.913906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b/users/157dfaa196bc42d5962961d24c698eac/roles/f6ce35b028e548f68db97d1f5bfc955e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.917551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] RBAC: Authorizing `identity:create_grant(project_id=633f8860113345a89d56f3146f465a5b, user_id=157dfaa196bc42d5962961d24c698eac, role_id=f6ce35b028e548f68db97d1f5bfc955e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.920186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.935946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfe179-c0e5-4387-a07b-0654a640cfe7 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:07.947310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 145/290] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:07 2022] PUT /identity/v3/projects/633f8860113345a89d56f3146f465a5b/users/157dfaa196bc42d5962961d24c698eac/roles/f6ce35b028e548f68db97d1f5bfc955e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:07.976199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:07.988169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:07.989788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:07.990284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:07.990763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] PATH_INFO: `/v3/users/2a5350040a4e435c8f1e530bb39ff789` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:07.992441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5350040a4e435c8f1e530bb39ff789)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:07.994303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:07.996931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 146/291] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:07 2022] GET /identity/v3/users/2a5350040a4e435c8f1e530bb39ff789 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:08.003918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:08.082414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:08.125026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3f6c9-a60a-414c-91bf-3fc73600f986 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:08.164749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:08.165795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:08.166313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:08.166790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:08.245395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:08.247226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:08.313910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:08.351460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 146/292] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:08.357065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:08.492665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbad93d-a3a2-4b4b-aca6-20055c4c8229 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:08.552615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfff6fd-ba4b-4aa4-95c5-21dba097ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:08.553416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfff6fd-ba4b-4aa4-95c5-21dba097ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:08.553698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfff6fd-ba4b-4aa4-95c5-21dba097ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:08.553968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfff6fd-ba4b-4aa4-95c5-21dba097ac13 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:08.555371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfff6fd-ba4b-4aa4-95c5-21dba097ac13 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:08.556823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfff6fd-ba4b-4aa4-95c5-21dba097ac13 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:08.558873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 147/293] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:08 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:08.564269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:08.682967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225b483-f8bd-4cdb-a5bb-831e9ec86894 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:08.719848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:08.720504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:08.720791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:08.721069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:08.722459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=2a5350040a4e435c8f1e530bb39ff789) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:08.722774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:08.724311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bab60f-48be-4dc1-9fc1-5173a08e479c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:08.730758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 147/294] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:08 2022] GET /identity/v3/role_assignments?user.id=2a5350040a4e435c8f1e530bb39ff789&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:09.308124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4fd7f-26d1-4ab2-9495-73b74c3dc7e2 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.308640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4fd7f-26d1-4ab2-9495-73b74c3dc7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.308715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4fd7f-26d1-4ab2-9495-73b74c3dc7e2 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.310341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 148/295] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:09.316709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d3beb-2e90-4967-b7d9-cf6be8c7b61b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.316998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d3beb-2e90-4967-b7d9-cf6be8c7b61b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.317276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d3beb-2e90-4967-b7d9-cf6be8c7b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.336184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-465d3beb-2e90-4967-b7d9-cf6be8c7b61b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:09.337453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-465d3beb-2e90-4967-b7d9-cf6be8c7b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:09.366442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:09.413197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 148/296] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:09.437335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4fd7f-26d1-4ab2-9495-73b74c3dc7e2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:09.502877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:09.511120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab069fca-8a43-47b1-84a4-e1ad24087215 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.511376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab069fca-8a43-47b1-84a4-e1ad24087215 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.511655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab069fca-8a43-47b1-84a4-e1ad24087215 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.513156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 149/297] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:09.518024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.518311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.518593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.535308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:09.536487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:09.554613 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4fd7f-26d1-4ab2-9495-73b74c3dc7e2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:09.603272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:09.604779 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.605066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.605399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.617905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:09.619079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 150/298] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:09.625811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:09.626697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:09.627522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 149/299] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:09.633609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:09.639971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:09.737129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:09.748093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e89ec-18dd-4cfe-bd39-248c756bb195 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:09.759939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1121cf4e-31d7-4b3f-b1f4-bed8ccc88e6b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:09.783216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:09.783481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:09.783881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.784163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.785240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.786088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:09.786400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:09.787939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e77822-dc7b-4eb1-9b3f-8603aa2e177b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:09.796932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 151/300] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:09.798168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d360973-3461-42a8-b622-34278c0b04de None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.798466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d360973-3461-42a8-b622-34278c0b04de None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.798745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d360973-3461-42a8-b622-34278c0b04de None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.800686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 152/301] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:09.804956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259df140-33e6-46d3-9b53-4ab4a96ee34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:09.805399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d360973-3461-42a8-b622-34278c0b04de None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:09.805785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df140-33e6-46d3-9b53-4ab4a96ee34d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.806035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df140-33e6-46d3-9b53-4ab4a96ee34d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.806313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df140-33e6-46d3-9b53-4ab4a96ee34d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.807510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259df140-33e6-46d3-9b53-4ab4a96ee34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:09.808977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259df140-33e6-46d3-9b53-4ab4a96ee34d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:09.810594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 150/302] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:09.811771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.812058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.812336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.830589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:09.832264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:09.902494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 151/303] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:09.903250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:09.924409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d360973-3461-42a8-b622-34278c0b04de None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:09.959817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:09.960692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:09.960984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:09.961261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] PATH_INFO: `/v3/users/dd83dd368f9d4c50870a2cbbe87e2fb3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:09.975215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] RBAC: Authorizing `identity:get_user(user_id=dd83dd368f9d4c50870a2cbbe87e2fb3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:09.976661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:09.978487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 153/304] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:09 2022] GET /identity/v3/users/dd83dd368f9d4c50870a2cbbe87e2fb3 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:09.979093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:09.991105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:10.013559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a3c33-b223-4aeb-81be-4097e6182505 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:10.035340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:10.052546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:10.053220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.053508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.053786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] PATH_INFO: `/v3/groups` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.054473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:10.055957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:10.075070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 152/305] 10.222.0.61 () {60 vars in 1134 bytes} [Tue Jun 7 13:04:09 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:10.075070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:10.086614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce6fe2-7636-4b38-b0c8-c023f20bee44 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:10.128410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:10.129293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.129579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.129855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.143010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:10.152141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:10.153330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:10.154404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 154/306] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:10.161112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:10.228855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c60f20-c7f9-4737-8796-4a005c93abfe None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:10.248308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:10.281038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:10.281700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.281981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.282266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.283799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] RBAC: Adding query filter params (role.id=f6ce35b028e548f68db97d1f5bfc955e, user.id=dd83dd368f9d4c50870a2cbbe87e2fb3, scope.system=all) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:10.284122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:10.286154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:10.288126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e9b83-8476-4aaf-89c5-b5cd8553e567 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:10.292156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 153/307] 10.222.0.61 () {56 vars in 1308 bytes} [Tue Jun 7 13:04:10 2022] GET /identity/v3/role_assignments?user.id=dd83dd368f9d4c50870a2cbbe87e2fb3&scope.system=all&role.id=f6ce35b028e548f68db97d1f5bfc955e => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:10.327093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:10.328244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.328724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.329205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.330185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:10.330693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:10.332927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:10.345755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 155/308] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:10.352144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:10.406384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:10.472824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf9f19-39b9-492d-afc4-a4017bf9d01e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:10.508915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1db67-3470-41e1-b662-1dc299b587f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:10.509786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1db67-3470-41e1-b662-1dc299b587f1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.510068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1db67-3470-41e1-b662-1dc299b587f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.510340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1db67-3470-41e1-b662-1dc299b587f1 None admin] PATH_INFO: `/v3/users/071bdd43185844249bbecaf58c78f939` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.511540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1db67-3470-41e1-b662-1dc299b587f1 None admin] RBAC: Authorizing `identity:get_user(user_id=071bdd43185844249bbecaf58c78f939)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:10.512986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1db67-3470-41e1-b662-1dc299b587f1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:10.514769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 154/309] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:10 2022] GET /identity/v3/users/071bdd43185844249bbecaf58c78f939 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:10.520900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:10.625691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4f3bd-c5bc-4da6-a79e-5ef9573947c4 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:10.656344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:10.662655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d848077-f167-4944-a500-b87de48f061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:10.667328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d848077-f167-4944-a500-b87de48f061c None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.667777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d848077-f167-4944-a500-b87de48f061c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.668180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d848077-f167-4944-a500-b87de48f061c None admin] PATH_INFO: `/v3/system/users/071bdd43185844249bbecaf58c78f939/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.669843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d848077-f167-4944-a500-b87de48f061c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=071bdd43185844249bbecaf58c78f939, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:10.671385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d848077-f167-4944-a500-b87de48f061c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:10.682465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 156/310] 10.222.0.61 () {58 vars in 1334 bytes} [Tue Jun 7 13:04:10 2022] PUT /identity/v3/system/users/071bdd43185844249bbecaf58c78f939/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:10.767392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f554d55-647a-4660-8406-33a7e34f7503 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.767847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f554d55-647a-4660-8406-33a7e34f7503 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.768222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f554d55-647a-4660-8406-33a7e34f7503 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.770601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 155/311] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:10.775950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:10.776465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:10.776552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:10.794578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:10.795826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:10.873426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 157/312] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:10.897496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f554d55-647a-4660-8406-33a7e34f7503 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:11.004881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f554d55-647a-4660-8406-33a7e34f7503 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:11.017946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:11.038580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:11.039408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:11.039899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:11.040179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:11.051822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:11.059570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:11.060466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:11.061277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 156/313] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:11.068671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:11.172072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ead43b-b4de-44d4-a0db-c20bddfa3167 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:11.200736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:11.209987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:11.210857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:11.211280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:11.211719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:11.213020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:11.213994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:11.215570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:11.225470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 158/314] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:11.232203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:11.339407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f0eb7-c473-4a43-848e-74773ade02af None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:11.374596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:11.375505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:11.375924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:11.376199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] PATH_INFO: `/v3/users/157dfaa196bc42d5962961d24c698eac` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:11.377403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] RBAC: Authorizing `identity:get_user(user_id=157dfaa196bc42d5962961d24c698eac)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:11.378897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:11.380958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 157/315] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:11 2022] GET /identity/v3/users/157dfaa196bc42d5962961d24c698eac => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:11.386394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:11.501764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f39308-78ea-42f6-abf6-fb7c1d7a1d49 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:11.538452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:11.539417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:11.539878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:11.540323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:11.599307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:11.610792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:11.612465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:11.711791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 159/316] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:11.717420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:11.771399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:11.820019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f6453-931f-49b2-95af-9dd114c9f37e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:11.855335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977322-cfa7-4619-9299-c12378f9dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:11.856502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977322-cfa7-4619-9299-c12378f9dd0e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:11.856996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977322-cfa7-4619-9299-c12378f9dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:11.857467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977322-cfa7-4619-9299-c12378f9dd0e None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:11.858899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af977322-cfa7-4619-9299-c12378f9dd0e None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:11.860536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af977322-cfa7-4619-9299-c12378f9dd0e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:11.862790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 158/317] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:11 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:11.868743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:11.962108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:11.978900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1624b-90dc-4356-b8e8-7b4813240bdf None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:12.013274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:12.014168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.014592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.015033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.016000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] RBAC: Adding query filter params (role.id=f6ce35b028e548f68db97d1f5bfc955e, scope.project.id=633f8860113345a89d56f3146f465a5b, user.id=157dfaa196bc42d5962961d24c698eac) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:12.016476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:12.018047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56857aa7-5cde-4c8b-8294-0b7c31e34397 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:12.025014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 160/318] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:11 2022] GET /identity/v3/role_assignments?user.id=157dfaa196bc42d5962961d24c698eac&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=f6ce35b028e548f68db97d1f5bfc955e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:12.467730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7911db-02bd-4ab1-902a-93b7b2910d07 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.468248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7911db-02bd-4ab1-902a-93b7b2910d07 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.468325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7911db-02bd-4ab1-902a-93b7b2910d07 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.469810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 159/319] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:12.474956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95832f4f-7a4e-4c7a-94dc-9523881f7a13 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.475255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95832f4f-7a4e-4c7a-94dc-9523881f7a13 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.475537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95832f4f-7a4e-4c7a-94dc-9523881f7a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.492571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-95832f4f-7a4e-4c7a-94dc-9523881f7a13 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:12.493769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-95832f4f-7a4e-4c7a-94dc-9523881f7a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:12.563265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 161/320] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:12.586502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7911db-02bd-4ab1-902a-93b7b2910d07 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:12.645891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5823d4-9d28-45ac-bb8b-4d46d9bb8eb9 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.646206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5823d4-9d28-45ac-bb8b-4d46d9bb8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.646450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5823d4-9d28-45ac-bb8b-4d46d9bb8eb9 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.647864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 162/321] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:12.652582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.652869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.653148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.670072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:12.671291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:12.693651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7911db-02bd-4ab1-902a-93b7b2910d07 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:12.704793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:12.737187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:12.739339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.739634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.740056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.741762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:12.744064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:12.750624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 160/322] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:12.750856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 163/323] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:12.751975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:12.769185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:12.783333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:12.859280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb4cb-afa8-4b01-807e-db277d37d1f9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:12.873076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecf60a-5b0d-4e8c-985a-a75f22620bc3 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:12.895394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:12.896070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.896381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.896658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] PATH_INFO: `/v3/groups` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.897356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:12.898803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:12.913815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f329944-dc1d-401f-8478-487fa2069075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:12.914617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f329944-dc1d-401f-8478-487fa2069075 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:12.915032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f329944-dc1d-401f-8478-487fa2069075 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:12.922978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f329944-dc1d-401f-8478-487fa2069075 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:12.923162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 161/324] 10.222.0.61 () {60 vars in 1134 bytes} [Tue Jun 7 13:04:12 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:12.934737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f329944-dc1d-401f-8478-487fa2069075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9f329944-dc1d-401f-8478-487fa2069075 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:12.943354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:12.944361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:12.950403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 164/325] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:12.951992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:12.993373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:13.064592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aee582-7405-4dc4-9d11-f8ed8552aa43 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.072992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0cf4-0a1b-4660-89ed-68c86f406825 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.073277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0cf4-0a1b-4660-89ed-68c86f406825 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.073553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b0cf4-0a1b-4660-89ed-68c86f406825 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.074961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 165/326] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:13.080776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.081096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.081824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.099673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:13.100881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:13.105089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.105765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.106049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.106327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.107138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] RBAC: Adding query filter params (name=reader) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:13.107447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:13.108840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:13.118847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 162/327] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:13.124116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:13.177259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 166/328] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:13.207513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:13.210137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:13.242818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:13.248493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beb75f7-a0c2-48e4-9696-fce42ec77dce None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.292073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.292947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.293225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.293493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] PATH_INFO: `/v3/users/dd83dd368f9d4c50870a2cbbe87e2fb3` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.294681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] RBAC: Authorizing `identity:get_user(user_id=dd83dd368f9d4c50870a2cbbe87e2fb3)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:13.298925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:13.303374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 163/329] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:13 2022] GET /identity/v3/users/dd83dd368f9d4c50870a2cbbe87e2fb3 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:13.305637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:13.333967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ccbe-54e9-4619-b05b-51a72ac36bfe None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.374701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539db545-e65b-432c-8436-0371b3521339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.375655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539db545-e65b-432c-8436-0371b3521339 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.375945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539db545-e65b-432c-8436-0371b3521339 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.376281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539db545-e65b-432c-8436-0371b3521339 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.388176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539db545-e65b-432c-8436-0371b3521339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-539db545-e65b-432c-8436-0371b3521339 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:13.397890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:13.398815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:13.399656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:13.403101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 167/330] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:13.405302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539db545-e65b-432c-8436-0371b3521339 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:13.435425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3e927-7acf-4efc-be11-de6de20061a2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.481055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.481865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.482148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.482424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] PATH_INFO: `/v3/system/users/dd83dd368f9d4c50870a2cbbe87e2fb3/roles/f6ce35b028e548f68db97d1f5bfc955e` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.484034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dd83dd368f9d4c50870a2cbbe87e2fb3, role_id=f6ce35b028e548f68db97d1f5bfc955e)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:13.485438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:13.488805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:13.497474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 164/331] 10.222.0.61 () {58 vars in 1334 bytes} [Tue Jun 7 13:04:13 2022] PUT /identity/v3/system/users/dd83dd368f9d4c50870a2cbbe87e2fb3/roles/f6ce35b028e548f68db97d1f5bfc955e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:13.540168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539db545-e65b-432c-8436-0371b3521339 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.584567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.585550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.586011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.586492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.587470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:13.587956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:13.589514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:13.599666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 168/332] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:13.611011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:13.655231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:13.743064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679cf3f-4bb6-47ed-a006-ee1b501f6cbf None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.787023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1771e-16f1-4c34-bae5-8af94feda746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.787916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1771e-16f1-4c34-bae5-8af94feda746 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.788195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1771e-16f1-4c34-bae5-8af94feda746 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.788470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1771e-16f1-4c34-bae5-8af94feda746 None admin] PATH_INFO: `/v3/users/071bdd43185844249bbecaf58c78f939` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.790199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1771e-16f1-4c34-bae5-8af94feda746 None admin] RBAC: Authorizing `identity:get_user(user_id=071bdd43185844249bbecaf58c78f939)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:13.791669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1771e-16f1-4c34-bae5-8af94feda746 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:13.793330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 165/333] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:13 2022] GET /identity/v3/users/071bdd43185844249bbecaf58c78f939 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:13.799160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:13.946054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca304f39-59c1-427a-99cb-56fd39428bef None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:13.970315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:13.982127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:13.982787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:13.983103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:13.983381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:13.984222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] RBAC: Adding query filter params (role.id=2bbf1931a8df4f6da54f4d30d99aa430, user.id=071bdd43185844249bbecaf58c78f939, scope.system=all) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:13.984535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:13.985984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94627fc9-aa86-477f-bd12-9123d4ef3253 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:13.999025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 169/334] 10.222.0.61 () {56 vars in 1308 bytes} [Tue Jun 7 13:04:13 2022] GET /identity/v3/role_assignments?user.id=071bdd43185844249bbecaf58c78f939&scope.system=all&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:15.739421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aa813-2cf3-4b09-bfc8-786dc831c018 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:15.739963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aa813-2cf3-4b09-bfc8-786dc831c018 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:15.739963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aa813-2cf3-4b09-bfc8-786dc831c018 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:15.741583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 166/335] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:15.749113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:15.749564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:15.749645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:15.775884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:15.777094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:15.859445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 170/336] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:15.896070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aa813-2cf3-4b09-bfc8-786dc831c018 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:16.013619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:16.019486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aa813-2cf3-4b09-bfc8-786dc831c018 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:16.059877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:16.060679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.060958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.061238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.077575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:16.085943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:16.086930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:16.088655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 167/337] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:16.093276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:16.227140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32839d-f036-4796-adf7-a559e208a683 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:16.243783 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:16.269660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:16.270685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.271214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.271697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.272664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:16.272955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:16.274336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358be6c-b666-44ed-9942-472e0a351054 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:16.286765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 171/338] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:16.293003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:16.428452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd56704-fd0f-4317-9f3b-d9080e051c8c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:16.468988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a5a56-f693-452c-8eb0-adb6690a22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:16.469650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5a56-f693-452c-8eb0-adb6690a22d1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.469928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5a56-f693-452c-8eb0-adb6690a22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.470202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a5a56-f693-452c-8eb0-adb6690a22d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.477333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d8217-bc78-4b12-aa65-8804521048fa None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.477638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d8217-bc78-4b12-aa65-8804521048fa None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.477885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d8217-bc78-4b12-aa65-8804521048fa None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.479563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 172/339] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:16.491285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.491806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.492180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.517362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:16.520398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:16.521765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:16.600456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a5a56-f693-452c-8eb0-adb6690a22d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:16.601929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a5a56-f693-452c-8eb0-adb6690a22d1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:16.640759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 173/340] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:16.676334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:16.756113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 168/341] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:16.758268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b24cf8-dd9d-4558-b1a4-dc5918eec5e9 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.760132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b24cf8-dd9d-4558-b1a4-dc5918eec5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.760710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b24cf8-dd9d-4558-b1a4-dc5918eec5e9 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.769819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 169/342] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:16.771043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b24cf8-dd9d-4558-b1a4-dc5918eec5e9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:16.858811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0eec8-521f-4008-bc9c-ff2be550be7d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:16.864963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:16.887207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:16.935211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59645e-6fb7-4cc6-82ee-6efa12e18833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:16.936030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59645e-6fb7-4cc6-82ee-6efa12e18833 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.936312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59645e-6fb7-4cc6-82ee-6efa12e18833 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.936583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59645e-6fb7-4cc6-82ee-6efa12e18833 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:16.954954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b24cf8-dd9d-4558-b1a4-dc5918eec5e9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:16.958665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59645e-6fb7-4cc6-82ee-6efa12e18833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7e59645e-6fb7-4cc6-82ee-6efa12e18833 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:16.968763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:16.969690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:16.970949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 174/343] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:16.971916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:16.972613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:16.972785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.000193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:17.002034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:17.006660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53466598-e687-4eb9-be7b-30becd7c74f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.007611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466598-e687-4eb9-be7b-30becd7c74f9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.007892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466598-e687-4eb9-be7b-30becd7c74f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.008168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466598-e687-4eb9-be7b-30becd7c74f9 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.009375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53466598-e687-4eb9-be7b-30becd7c74f9 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:17.011400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53466598-e687-4eb9-be7b-30becd7c74f9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:17.013494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 170/344] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:16 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:17.015104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6b8b-a3ce-4e81-84a8-6d29912a1c1b None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.015392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6b8b-a3ce-4e81-84a8-6d29912a1c1b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.015669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6b8b-a3ce-4e81-84a8-6d29912a1c1b None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.016968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 171/345] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:17.017656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6b8b-a3ce-4e81-84a8-6d29912a1c1b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.088878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 175/346] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:17.089688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.141377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6b8b-a3ce-4e81-84a8-6d29912a1c1b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.169856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:17.184688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.185367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.185646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.185922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.186776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:17.187116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:17.188483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fa6d1-19c9-4338-8a6c-9726dad9c601 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:17.193134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:17.199918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f9fa-e0dd-4ea6-9aa6-6e5e8383171f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.200523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 172/347] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:17.201906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.202333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.202776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.226979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:17.228423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:17.245694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.246695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.247021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.247301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.255927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:17.257331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:17.259487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 176/348] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:17.260140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.337329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 173/349] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:17.338326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.400581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7425d-836e-4a64-97ab-72326e99cd6d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.447025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.447827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.448111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.448384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.461528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:17.467561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae1af2-95d9-4e2e-8440-df910058b571 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:17.469494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:17.470666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 177/350] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:17.471718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.495587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:17.498139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:17.521432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.522424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.522950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.523470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.586076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b6683-5fb6-4f9c-9707-c933c9d79535 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.603166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:17.605095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:17.621468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.622124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.622415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.622730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.623594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:17.623903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:17.625360 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:17.632372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 178/351] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:17.633199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.724781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:17.727108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 174/352] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:17.731813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.747529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203893a3-f34c-420b-b2c8-f7610e5c5f4e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.762161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:17.786701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315c0e-f765-453b-ad59-26537ff24aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.787553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315c0e-f765-453b-ad59-26537ff24aac None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.787834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315c0e-f765-453b-ad59-26537ff24aac None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.788108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315c0e-f765-453b-ad59-26537ff24aac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.801591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315c0e-f765-453b-ad59-26537ff24aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-6d315c0e-f765-453b-ad59-26537ff24aac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:17.810603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:17.811600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:17.812405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 179/353] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:17.813053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6380e-873b-470a-8316-44547f7199ed None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.813118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6380e-873b-470a-8316-44547f7199ed None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.813722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6380e-873b-470a-8316-44547f7199ed None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.814968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 180/354] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:17.818097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6380e-873b-470a-8316-44547f7199ed None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.885910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4c422-8f00-4a6d-af2b-27596521c044 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.935720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.936396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.936826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.937101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:17.939023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:17.939332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:17.942537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:17.946876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6380e-873b-470a-8316-44547f7199ed None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:17.952983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 175/355] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:17.955656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:17.998401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07ed35-1228-4d53-82ca-633752725a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:17.999342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07ed35-1228-4d53-82ca-633752725a13 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:17.999625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07ed35-1228-4d53-82ca-633752725a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:17.999898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07ed35-1228-4d53-82ca-633752725a13 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.001493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07ed35-1228-4d53-82ca-633752725a13 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.003397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07ed35-1228-4d53-82ca-633752725a13 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.005884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 181/356] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.007201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.007859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.008137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.028463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:18.030522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:18.035032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.047327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.072184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151dc7c2-b3fd-4691-a64b-6b8285dc73e3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.117018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 182/357] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:18.117867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.120747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.122313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.122599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.122896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.123655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:18.123958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.125338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.137950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 176/358] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.138888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.233854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481992d-8a8a-4501-a8dd-45303234e731 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.265752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda6564-66aa-4d83-996a-fc7f8b9970a1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.282041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.282957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.283247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.283526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] PATH_INFO: `/v3/users/dd83dd368f9d4c50870a2cbbe87e2fb3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.285353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] RBAC: Authorizing `identity:get_user(user_id=dd83dd368f9d4c50870a2cbbe87e2fb3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.286818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.288617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 183/359] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/users/dd83dd368f9d4c50870a2cbbe87e2fb3 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.289394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.303022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.303888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.304168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.304467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.305312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.306594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.308095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.309936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 177/360] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.310533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.328878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.429741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6687-5078-414b-9fd0-6a74bddecacb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.437538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c6a07-0e3b-4d79-bdf2-9a6c804c0488 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.470007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.470816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.471145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.471712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.479746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993a1af-b7d2-4c3f-b6ec-92ea1c06dbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.480569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993a1af-b7d2-4c3f-b6ec-92ea1c06dbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.480839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993a1af-b7d2-4c3f-b6ec-92ea1c06dbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.481144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993a1af-b7d2-4c3f-b6ec-92ea1c06dbd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.495647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993a1af-b7d2-4c3f-b6ec-92ea1c06dbd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-7993a1af-b7d2-4c3f-b6ec-92ea1c06dbd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:18.503866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:18.504912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:18.505708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:18.508357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 178/361] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:18.511335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063af46-358f-445f-be99-efd08c37ce6d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.511851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063af46-358f-445f-be99-efd08c37ce6d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.512325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063af46-358f-445f-be99-efd08c37ce6d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.514126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 179/362] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:18.515143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063af46-358f-445f-be99-efd08c37ce6d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.564990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.566749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.577518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.602169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.649628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063af46-358f-445f-be99-efd08c37ce6d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.691377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 184/363] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:18.692466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.697565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.698250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.698532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.698806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.699676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] RBAC: Adding query filter params (role.id=f6ce35b028e548f68db97d1f5bfc955e, user.id=dd83dd368f9d4c50870a2cbbe87e2fb3, scope.system=all) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:18.699986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.701431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.719936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 180/364] 10.222.0.61 () {56 vars in 1308 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/role_assignments?user.id=dd83dd368f9d4c50870a2cbbe87e2fb3&scope.system=all&role.id=f6ce35b028e548f68db97d1f5bfc955e => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.722319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:18.819914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56d216-d06a-4326-acbf-d2585d29f485 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.831186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.865898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.866705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.866898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.867341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.868335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:18.868824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.871435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638cd92-f745-4dc8-955a-95a11c51586e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.878812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b47272-c2ed-43b8-b091-678168873e48 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:18.883154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 185/365] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.884684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.884975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.885251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.914676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:18.922807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:18.925230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:18.932488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:18.933187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:18.933465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:18.933737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:18.934562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:18.934902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:18.936377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:18.954570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 181/366] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:18.955793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.033565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 186/367] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:19.035408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.089697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ff218-1d43-40f1-8fd2-dede79c80704 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.138740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.139892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.141410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.142538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.144992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.146798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.149874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 182/368] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:18 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:19.150910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.198715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a6d16-f036-4add-96dc-bfe35f799ccf None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.200490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.238673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.239407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.239584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.239703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.239957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.294654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e5466-1697-482d-a924-4156f32c7238 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.327761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.329363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.348900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.350040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.350517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.351032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.363634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:19.371625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:19.372495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:19.373321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:19.375744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 183/369] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:19.376837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.449904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 187/370] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:19.454853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.454853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.487209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3ea17-224f-4c9e-9cff-26f6a3a9cb31 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.505947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.533147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.534940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.535518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.535994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.537494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.539269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.541200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 184/371] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:19.542160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.559929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116eb4b-49de-49d9-ac99-20b149306fe0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.600235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.600919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.601273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.601616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.602450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:19.604037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.604811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.617179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 188/372] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:19.618263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.674889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbca113-50e0-45ad-81d7-a02480638053 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.689903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.723469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.724350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.724640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.724991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.726176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.727606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.731911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 185/373] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:19.732696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f86fe3-0d15-4bd4-bbd1-3646d27a1e07 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.732806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.770332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.771380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.771841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.772703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.773832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=633f8860113345a89d56f3146f465a5b) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:19.774317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.775978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.782452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 189/374] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:19.783654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:19.803935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.858028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dccbf12-4d2d-47ee-a842-93eab42a0f01 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.905181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b072b2-2b39-426b-b21e-f64c37fc92e8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:19.916158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae418a-ddd4-4716-9d47-3818cb0db697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.916731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae418a-ddd4-4716-9d47-3818cb0db697 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.916731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae418a-ddd4-4716-9d47-3818cb0db697 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.916959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae418a-ddd4-4716-9d47-3818cb0db697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.947344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:19.955903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:19.957740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:19.958239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:19.958700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:19.961913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:19.963711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:19.965952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 190/375] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:19.974231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:20.044977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae418a-ddd4-4716-9d47-3818cb0db697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:20.047110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae418a-ddd4-4716-9d47-3818cb0db697 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:20.125634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:20.146014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351dc52c-a406-40fc-9a7a-af5f0cb7c9a2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:20.193395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:20.194404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.194881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.195331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.196356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=633f8860113345a89d56f3146f465a5b) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:20.197744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:20.201131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:20.213346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 191/376] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:20.230988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 186/377] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:20.232149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:20.337860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:20.396347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2c288-e8d8-4a80-9c67-a2422b6a8a53 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:20.412805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142a70-6bbb-4e12-abc1-51147400b93a None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.413223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142a70-6bbb-4e12-abc1-51147400b93a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.413571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142a70-6bbb-4e12-abc1-51147400b93a None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.417713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 187/378] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:20.428641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.428923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.429194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.448785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:20.450136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.450652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.451150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] PATH_INFO: `/v3/projects/f9260a652cb24f75867436268df3aa69` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.452702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=f9260a652cb24f75867436268df3aa69)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:20.454397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:20.457328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 192/379] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:20 2022] GET /identity/v3/projects/f9260a652cb24f75867436268df3aa69 => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:20.463133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:20.472063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:20.473290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:20.598053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e66e-3695-4e83-a649-c7e39ca41ba0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:20.620197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 188/380] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:20.636593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:20.647301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:20.648119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.648400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.648671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] PATH_INFO: `/v3/groups/395f398fd2b14371ad8308361e0ae5ba` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.652819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:20.656415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] RBAC: Authorizing `identity:get_group(group_id=395f398fd2b14371ad8308361e0ae5ba)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:20.657871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:20.666850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 193/381] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:20 2022] GET /identity/v3/groups/395f398fd2b14371ad8308361e0ae5ba => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:20.670811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:20.807235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:20.829855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f876d2a-9ebf-441d-a706-1925fcac5928 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:20.835797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9607d-3d7f-4fed-b8e8-0873566e23e2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:20.892985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:20.893641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.893946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.894221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.895077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] RBAC: Adding query filter params (group.id=395f398fd2b14371ad8308361e0ae5ba, role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=f9260a652cb24f75867436268df3aa69) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:20.895386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:20.896821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:20.904276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:20.905003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:20.905280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:20.905548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:20.911018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 194/382] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:20 2022] GET /identity/v3/role_assignments?group.id=395f398fd2b14371ad8308361e0ae5ba&scope.project.id=f9260a652cb24f75867436268df3aa69&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:20.920929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:20.929634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:20.930643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:20.931464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 189/383] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:20.938516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:21.017873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:21.071061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f3159-c8eb-4e20-a2c3-521e2ed1277a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:21.112391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:21.113695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:21.113971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:21.114246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:21.115028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:21.115332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:21.116732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:21.130582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 195/384] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:21.138178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:21.264011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280b0f-9be6-4a3d-a8ef-aeacc1b734a1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:21.311733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:21.312593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:21.312874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:21.313142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:21.314390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:21.316207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:21.318046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 190/385] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:21 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:21.324545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:21.453433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ca4bb-07e4-4a19-b613-1986d2f5cab5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:21.472669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:21.490316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9c2f7-11e0-4bf9-a8bb-29e394737830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:21.491206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9c2f7-11e0-4bf9-a8bb-29e394737830 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:21.491487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9c2f7-11e0-4bf9-a8bb-29e394737830 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:21.491755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9c2f7-11e0-4bf9-a8bb-29e394737830 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:21.492937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9c2f7-11e0-4bf9-a8bb-29e394737830 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:21.494376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9c2f7-11e0-4bf9-a8bb-29e394737830 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:21.496432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 196/386] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:21 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:21.503977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:21.525181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:21.628328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8969d73-08c4-4e07-8cde-b53e4d399b67 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:21.673570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6afe68-ef1e-4dfe-b044-ecdce0242380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:21.674461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6afe68-ef1e-4dfe-b044-ecdce0242380 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:21.674737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6afe68-ef1e-4dfe-b044-ecdce0242380 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:21.675075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6afe68-ef1e-4dfe-b044-ecdce0242380 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:21.680098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6afe68-ef1e-4dfe-b044-ecdce0242380 None admin] RBAC: Authorizing `identity:create_grant(project_id=33a909cd233e4d5abcb4773bc2a46e27, group_id=b85c424b423940aa8be4a4fc3746a98a, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:21.681550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6afe68-ef1e-4dfe-b044-ecdce0242380 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:21.705139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 191/387] 10.222.0.61 () {58 vars in 1442 bytes} [Tue Jun 7 13:04:21 2022] PUT /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:21.827754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a22f-eb63-469a-93b8-de5e51b73f17 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:21.828053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a22f-eb63-469a-93b8-de5e51b73f17 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:21.828217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5a22f-eb63-469a-93b8-de5e51b73f17 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:21.829996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 197/388] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:21.840823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:21.841111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:21.841368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:21.864046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:21.865277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:21.959552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 192/389] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:21.985236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a22f-eb63-469a-93b8-de5e51b73f17 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:22.104622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5a22f-eb63-469a-93b8-de5e51b73f17 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:22.147062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:22.147885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:22.148166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:22.148436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:22.160608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:22.168853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:22.169778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:22.170568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:22.172772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 198/390] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:22.180619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:22.182933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:22.321198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0fdea-851a-448c-999b-c64040a3f71b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:22.360401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:22.361010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:22.361290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:22.361562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:22.362333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:22.363194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:22.364628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:22.374479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 193/391] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:22.380849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:22.475963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:22.546588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf0cba-23c5-40e9-bcdf-304250b7d886 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:22.582579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa2b36-46e9-491a-9aa0-b548de289601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:22.583470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa2b36-46e9-491a-9aa0-b548de289601 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:22.583761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa2b36-46e9-491a-9aa0-b548de289601 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:22.584024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa2b36-46e9-491a-9aa0-b548de289601 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:22.585196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa2b36-46e9-491a-9aa0-b548de289601 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:22.586651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa2b36-46e9-491a-9aa0-b548de289601 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:22.588624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 199/392] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:22 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:22.599246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:22.670325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:22.722703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bf65e-b981-48b5-b220-f6a56ac19b8d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:22.733993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef227-c1c7-43e4-aa6c-2fab9f56e6cc None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:22.734337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef227-c1c7-43e4-aa6c-2fab9f56e6cc None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:22.734617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ef227-c1c7-43e4-aa6c-2fab9f56e6cc None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:22.736329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 200/393] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:22.743940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:22.744231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:22.744504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:22.774893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:22.776095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:22.782792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:22.783674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:22.783956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:22.784230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:22.787338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:22.788777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:22.790952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 194/394] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:22 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:22.797286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:22.918895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 201/395] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:22.971258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab4302-91a7-4b4a-b44d-d73b330b2950 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:23.006176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.020609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:23.031061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:23.031938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.032217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.032489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.038421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] RBAC: Authorizing `identity:create_grant(project_id=33a909cd233e4d5abcb4773bc2a46e27, group_id=b85c424b423940aa8be4a4fc3746a98a, role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:23.039901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:23.065805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 195/396] 10.222.0.61 () {58 vars in 1442 bytes} [Tue Jun 7 13:04:22 2022] PUT /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:04:23.091101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:23.162953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fa6db-92dd-419d-a905-fea2f327c2eb None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:23.205622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:23.207090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.207374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.207644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.223120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:23.231095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:23.232641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:23.233929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 202/397] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:23.245295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.365868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6662a6d-6778-4729-bf69-182dc89c617e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:23.404384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:23.405072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.405353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.405626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.406398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:23.406704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:23.408160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2de6c-8e64-4b64-9079-f585bdeb1e19 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:23.418461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 196/398] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:23.423890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.480236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16378e-3751-4b92-b77f-5a7edb3a06d9 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.480529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16378e-3751-4b92-b77f-5a7edb3a06d9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.480712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16378e-3751-4b92-b77f-5a7edb3a06d9 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.482211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 197/399] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:23.487180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.487464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.487780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.506202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:23.507477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:23.573537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b95bf-5cc7-4672-a54f-c5baee35cd18 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:23.579121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:23.582375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:23.619214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 198/400] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:23.636665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:23.637530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.637859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.638136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.643121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:23.644658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:23.646677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 203/401] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:23 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:23.651900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.688333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.844318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a31a4-e730-471a-8c2c-06e409b95538 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:23.852586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5efd8f1-99ee-4660-bc8b-9cd706f4767e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:23.892484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:23.893286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.893567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.893880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.902061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:23.902960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:23.903241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:23.903513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:23.904692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:23.905760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:23.906144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:23.908167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 199/402] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:23 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:23.918529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:23.919481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:23.920283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 204/403] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:23.922119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.931822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:23.981789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:23.988557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:24.061103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8e6af-d6cf-4dac-aa25-9887feeccbf5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.095285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a38e-37d4-4756-87e9-a5dc44c7ea66 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.106203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ebcdf-ead1-4bc8-b6f0-d21a2dbb6637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.107119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ebcdf-ead1-4bc8-b6f0-d21a2dbb6637 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.107398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ebcdf-ead1-4bc8-b6f0-d21a2dbb6637 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.107691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ebcdf-ead1-4bc8-b6f0-d21a2dbb6637 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.110548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ebcdf-ead1-4bc8-b6f0-d21a2dbb6637 None admin] RBAC: Authorizing `identity:create_grant(project_id=633f8860113345a89d56f3146f465a5b, group_id=b85c424b423940aa8be4a4fc3746a98a, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:24.112566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ebcdf-ead1-4bc8-b6f0-d21a2dbb6637 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:24.134938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 200/404] 10.222.0.61 () {58 vars in 1442 bytes} [Tue Jun 7 13:04:23 2022] PUT /identity/v3/projects/633f8860113345a89d56f3146f465a5b/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:24.136847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0556e-f38e-4690-bda8-3a41d8ff559a None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.137361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0556e-f38e-4690-bda8-3a41d8ff559a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.137885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0556e-f38e-4690-bda8-3a41d8ff559a None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.140999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 201/405] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:24.144429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.146654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.147223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.148060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.148518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.148794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.149774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.151149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:24.151658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:24.155465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:24.174065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 205/406] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:24.181851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:24.185943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:24.188227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:24.317224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:24.324742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce0c13-1563-4bd2-bb8d-8ed9e68c9ec5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.356320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 202/407] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:24.363296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.364640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.365099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.365549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.366994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:24.368652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:24.370515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 206/408] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:24 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:24.376422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:24.378701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:24.435757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:24.511032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe56a86-856d-4c82-b583-6295fcae6841 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.521055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bae88a-d83b-4364-a1b0-6d5652be1c7b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.549302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.550707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.555033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.556049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.557272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:24.558716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:24.560464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 203/409] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:24 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:24.566963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:24.568412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.569205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.569484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.569773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.592709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:24.606283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:24.607268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:24.608079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 207/410] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:24.616078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:24.689664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:24.699564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518ac85-3799-4955-88e8-55a71aebdc92 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.712023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:24.740984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31508fae-04d2-4373-a1e5-dbba2ed40b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.742383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31508fae-04d2-4373-a1e5-dbba2ed40b3f None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.743924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31508fae-04d2-4373-a1e5-dbba2ed40b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.745152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31508fae-04d2-4373-a1e5-dbba2ed40b3f None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.749733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31508fae-04d2-4373-a1e5-dbba2ed40b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=633f8860113345a89d56f3146f465a5b, group_id=b85c424b423940aa8be4a4fc3746a98a, role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:24.751600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31508fae-04d2-4373-a1e5-dbba2ed40b3f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:24.776816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 204/411] 10.222.0.61 () {58 vars in 1442 bytes} [Tue Jun 7 13:04:24 2022] PUT /identity/v3/projects/633f8860113345a89d56f3146f465a5b/groups/b85c424b423940aa8be4a4fc3746a98a/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:24.793772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca5f17-7bb3-4b3b-9574-c7680ffabeeb None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.794442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca5f17-7bb3-4b3b-9574-c7680ffabeeb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.795020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca5f17-7bb3-4b3b-9574-c7680ffabeeb None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.797177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 205/412] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:24.808569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.809186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.809706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.839895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:24.843368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:24.852649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1ab1d-351d-45ff-a6e3-5a47d1de6e6c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:24.917866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:24.923152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:24.923696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:24.924168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:24.925202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:24.925789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:24.927602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:24.938268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 208/413] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:24.948570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:24.996484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 206/414] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:25.021210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:25.061720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:25.083712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebdc67-339d-45ff-bc63-ba290e3a7a13 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.142474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:25.147040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64650387-355d-47b5-8c31-2c2c0955d95f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.165294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.166153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.166432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.166706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] PATH_INFO: `/v3/groups/395f398fd2b14371ad8308361e0ae5ba` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.168604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] RBAC: Authorizing `identity:get_group(group_id=395f398fd2b14371ad8308361e0ae5ba)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:25.169979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:25.171987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 209/415] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:24 2022] GET /identity/v3/groups/395f398fd2b14371ad8308361e0ae5ba => generated 229 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:25.177998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:25.215594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.216414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.216703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.216982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.228260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:25.236238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:25.237151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:25.238746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 207/416] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:25.243666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:25.353358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0199e7-3ba4-45bf-aa29-23f019ec9249 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.369013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ccba7-260b-4275-9e37-928a98e98922 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.391690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.392645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.392932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.393210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.394164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:25.394620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:25.396140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:25.405867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 208/417] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:25.411469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:25.413730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.414596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.414903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.415183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] PATH_INFO: `/v3/projects/f9260a652cb24f75867436268df3aa69` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.416390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] RBAC: Authorizing `identity:get_project(project_id=f9260a652cb24f75867436268df3aa69)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:25.417837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:25.419935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 210/418] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:25 2022] GET /identity/v3/projects/f9260a652cb24f75867436268df3aa69 => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:25.425843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:25.451257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:25.453581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:25.538787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c0988-7993-42ad-a6a0-7e395e43df6d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.539863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debac5e8-1998-489c-a1e7-fab8631153f1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.580458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.581531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.582130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.582614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.587394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.588086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.588368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.588642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] PATH_INFO: `/v3/projects/f9260a652cb24f75867436268df3aa69/groups/395f398fd2b14371ad8308361e0ae5ba/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.590700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9260a652cb24f75867436268df3aa69, group_id=395f398fd2b14371ad8308361e0ae5ba, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:25.592170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:25.611977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 211/419] 10.222.0.61 () {58 vars in 1442 bytes} [Tue Jun 7 13:04:25 2022] PUT /identity/v3/projects/f9260a652cb24f75867436268df3aa69/groups/395f398fd2b14371ad8308361e0ae5ba/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:04:25.664662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:25.666567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:25.695233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:25.781069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 209/420] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:04:25.787404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:25.895847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:25.916794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e9301-6679-4e8e-a967-b31c61b56fe5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:25.954325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8bdb8-8b16-4bb2-a952-63dd34c490b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:25.955593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8bdb8-8b16-4bb2-a952-63dd34c490b7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:25.956072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8bdb8-8b16-4bb2-a952-63dd34c490b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:25.957073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8bdb8-8b16-4bb2-a952-63dd34c490b7 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:25.958463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8bdb8-8b16-4bb2-a952-63dd34c490b7 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:25.960112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8bdb8-8b16-4bb2-a952-63dd34c490b7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:25.962341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 212/421] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:25 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:25.967508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:26.009475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd969a78-a4dc-4c6e-98a6-4b77998c180a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.010119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd969a78-a4dc-4c6e-98a6-4b77998c180a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.010591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd969a78-a4dc-4c6e-98a6-4b77998c180a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.012577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 213/422] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:26.018733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.019809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.020364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.039730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:26.041293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:26.073252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90385c5-06c9-4945-8662-fa5ee45ab92c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:26.098394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:26.120597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 214/423] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:26.126591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:26.127754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.128269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.128765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.130179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:26.131911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:26.133983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 210/424] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:25 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:26.139513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:26.199943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:26.266495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:26.288585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fb5cf-2773-4ad9-98f3-e4427441f903 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:26.342180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:26.342854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.343136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.343403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.344239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:26.344543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:26.346227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:26.361397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 215/425] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:26 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:26.407345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d9292-2525-468b-9d4b-bbdb10d6ac5c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:26.475707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:26.476555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.476837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.477106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.492619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:26.501041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:26.502020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:26.502817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:26.507213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 211/426] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:26.508949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:26.645708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837935ed-4cc8-4054-a489-32c03eeff0e3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:26.692308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:26.693102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.693383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.693655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.694411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:26.694714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:26.696302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31976f1-9599-4171-b551-d8ab251caaf7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:26.706205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 216/427] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:26.711745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:26.776451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:26.835784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734db8b6-100a-4fa9-ab90-d1f6b280a7cc None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:26.874173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:26.874897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.875189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.875469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.960648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:26.962060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:26.980449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d1cf-b0de-47da-8773-04361b1123ae None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.980737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d1cf-b0de-47da-8773-04361b1123ae None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.981022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0d1cf-b0de-47da-8773-04361b1123ae None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.983024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 217/428] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:26.988271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:26.988562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:26.988837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:26.999700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.017449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:27.018634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:27.027100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.102939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 212/429] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:27.107595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.120818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 218/430] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:27.151194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.229345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aebdcc-d959-4ddc-832f-be4707eebb4b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.268485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.269918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.270203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.270552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] PATH_INFO: `/v3/projects/33a909cd233e4d5abcb4773bc2a46e27` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.271801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] RBAC: Authorizing `identity:get_project(project_id=33a909cd233e4d5abcb4773bc2a46e27)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:27.273319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:27.275396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 213/431] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/projects/33a909cd233e4d5abcb4773bc2a46e27 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:27.281069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.282980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f784e24-a9ad-4020-af73-e530aa4ab777 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.308936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.334068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.334973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.335263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.335542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.338368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.351320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:27.363059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:27.364004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 07 13:04:27.364801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:27.365887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 219/432] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:27.372438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.408771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8bfc7-5fab-49e3-a429-509e6003e312 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.454178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.455357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.455638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.455911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.457155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:27.459050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:27.461353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 214/433] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:27.466675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.522592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a9c72-dd51-446a-a378-51823e0fc709 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.557872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.561796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.562457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.562738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.563039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.563811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] RBAC: Adding query filter params (name=member) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:27.564110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:27.565479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946c0d8-a599-487a-9aaa-fefcfaa9c158 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:27.574531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 220/434] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:27.575579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e7d11-a01f-4f6f-81bd-b356ee2fee67 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.575853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95617-2e9d-4c45-8547-eb37074729df None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.576143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95617-2e9d-4c45-8547-eb37074729df None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.576416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95617-2e9d-4c45-8547-eb37074729df None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.577763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 221/435] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:27.579658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff95617-2e9d-4c45-8547-eb37074729df None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.611239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.611895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.612384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.612493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.613839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=33a909cd233e4d5abcb4773bc2a46e27) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:27.614094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:27.615553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd7f6d-4bfc-43e6-bf62-f10617ecf187 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:27.622522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 215/436] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=33a909cd233e4d5abcb4773bc2a46e27&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:27.623549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.623835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.624112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.646258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:27.647742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:27.657405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.719227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff95617-2e9d-4c45-8547-eb37074729df None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.740526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 216/437] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:27.756970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.757793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.758174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.758552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.764137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.840418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:27.841821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:27.856920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.879747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed8fda5-65c0-4a9b-81bd-22ecd19a37c3 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:27.919215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:27.921956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:27.922872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:27.923146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:27.923537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:27.937542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:27.946108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:27.947223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:27.948117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 217/438] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:27.954396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:27.981535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 222/439] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:27.987268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.085399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb4db6-cf87-4fb3-b2d1-e947c471c647 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.121264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f19e-4bbd-44f4-a79e-6d96523cf91d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.130009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.131275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.131275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.131458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.132783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:28.132783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.133768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.145177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 218/440] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:28.151092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.153842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.166297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.167515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.167996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.168517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.170116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.171816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.174228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 223/441] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:27 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:28.180525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.207368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.263029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429b113-615f-466a-be61-028cdc0c8633 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.284188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007492dd-eb43-40ea-9fef-988cf2f1e852 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.298452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0e8dc-6987-4fd4-9615-85ab6dd34d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.299137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0e8dc-6987-4fd4-9615-85ab6dd34d72 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.299425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0e8dc-6987-4fd4-9615-85ab6dd34d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.299704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0e8dc-6987-4fd4-9615-85ab6dd34d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.323986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.325010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.325295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.325572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.326790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.328256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.330186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 224/442] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:28.335598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.378438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0e8dc-6987-4fd4-9615-85ab6dd34d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.379937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0e8dc-6987-4fd4-9615-85ab6dd34d72 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.382179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.443256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.452820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9c6ce-0ee7-44b8-a533-c16622acbe63 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.486167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 219/443] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:04:28.487389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.488046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.488328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.488603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.489548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2bbf1931a8df4f6da54f4d30d99aa430, scope.project.id=633f8860113345a89d56f3146f465a5b) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:28.489860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.491369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434954e-d314-4267-8c66-61b697686c71 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.491991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cb565-45e4-4568-b422-a327d9c87be6 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.492282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cb565-45e4-4568-b422-a327d9c87be6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.492559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958cb565-45e4-4568-b422-a327d9c87be6 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.493858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 220/444] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:28.494552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cb565-45e4-4568-b422-a327d9c87be6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.497304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 225/445] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2bbf1931a8df4f6da54f4d30d99aa430 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:28.499022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.499378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.499654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.517673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:28.519459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:28.596302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 226/446] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:28.621545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.625851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958cb565-45e4-4568-b422-a327d9c87be6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.627723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.666234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.667395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.667683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.667961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] PATH_INFO: `/v3/projects/633f8860113345a89d56f3146f465a5b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.669221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] RBAC: Authorizing `identity:get_project(project_id=633f8860113345a89d56f3146f465a5b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.670693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.672506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 221/447] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/projects/633f8860113345a89d56f3146f465a5b => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:28.678157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.737706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.757606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bedbf-3e92-49c0-bb3c-36fd73b50a73 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.801005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.801911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.802262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.802606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.807698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4a471-fef7-49f3-a54a-5b88a06e99e3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.817276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:28.825025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:28.825939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:28.826765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 227/448] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:28.832613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.847034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.847896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.848179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.848455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] PATH_INFO: `/v3/groups/b85c424b423940aa8be4a4fc3746a98a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.850492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] RBAC: Authorizing `identity:get_group(group_id=b85c424b423940aa8be4a4fc3746a98a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.852114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.854044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 222/449] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/groups/b85c424b423940aa8be4a4fc3746a98a => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:28.860618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:28.907763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:28.942870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b388f-cf7c-49c1-9774-333cc2666d20 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.967314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07509c20-7bc2-449b-955b-a11f4e12f6c7 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:28.976226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:28.976888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:28.977175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:28.977456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:28.978237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:28.978549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:28.979966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:28.988042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 228/450] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:28.993477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:29.001012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:29.001671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:29.001956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:29.002231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:29.003094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] RBAC: Adding query filter params (group.id=b85c424b423940aa8be4a4fc3746a98a, role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=633f8860113345a89d56f3146f465a5b) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:29.004010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:29.004548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:29.005492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:29.013934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 223/451] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/role_assignments?group.id=b85c424b423940aa8be4a4fc3746a98a&scope.project.id=633f8860113345a89d56f3146f465a5b&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:29.099950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5ad8-46b0-410d-b354-c7401ddde44e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:29.135348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:29.136004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:29.136288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:29.136565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:29.209836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:29.211255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:29.221325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:29.314414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 229/452] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:04:29.321057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:29.425007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b68d6-c55f-45d0-98d6-7b11b78f7e6d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:29.427093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:29.460889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:29.461685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:29.461964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:29.462235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] PATH_INFO: `/v3/projects/f9260a652cb24f75867436268df3aa69` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:29.463427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] RBAC: Authorizing `identity:get_project(project_id=f9260a652cb24f75867436268df3aa69)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:29.464860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:29.466543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 224/453] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:04:29 2022] GET /identity/v3/projects/f9260a652cb24f75867436268df3aa69 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:29.472455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:29.569177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1dba7-0f4d-45b4-b2f9-deb6390c372b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:29.579684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:29.604747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803678f6-ed94-4455-89b1-1f876bcbf929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:29.605580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803678f6-ed94-4455-89b1-1f876bcbf929 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:29.605862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803678f6-ed94-4455-89b1-1f876bcbf929 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:29.606136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803678f6-ed94-4455-89b1-1f876bcbf929 None admin] PATH_INFO: `/v3/groups/395f398fd2b14371ad8308361e0ae5ba` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:29.607318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803678f6-ed94-4455-89b1-1f876bcbf929 None admin] RBAC: Authorizing `identity:get_group(group_id=395f398fd2b14371ad8308361e0ae5ba)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:29.608775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803678f6-ed94-4455-89b1-1f876bcbf929 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:29.610381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 230/454] 10.222.0.61 () {56 vars in 1180 bytes} [Tue Jun 7 13:04:29 2022] GET /identity/v3/groups/395f398fd2b14371ad8308361e0ae5ba => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:29.616861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:29.711892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1a68d-eb1b-48b0-bdb7-2a4c8a0b3132 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:29.744912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:29.745548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:29.745828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:29.746101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:29.746958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] RBAC: Adding query filter params (group.id=395f398fd2b14371ad8308361e0ae5ba, role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=f9260a652cb24f75867436268df3aa69) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:29.747271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:29.748696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6856c-3309-48c6-b734-56bd9f8821a1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:29.754431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 225/455] 10.222.0.61 () {56 vars in 1376 bytes} [Tue Jun 7 13:04:29 2022] GET /identity/v3/role_assignments?group.id=395f398fd2b14371ad8308361e0ae5ba&scope.project.id=f9260a652cb24f75867436268df3aa69&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:32.549703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124e3ba-2353-4342-8d1e-27961fa0bd57 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.550768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124e3ba-2353-4342-8d1e-27961fa0bd57 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.550861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7124e3ba-2353-4342-8d1e-27961fa0bd57 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.552505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 231/456] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:32.557646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.558072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.558238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.579887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:32.581115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:32.614121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:32.669158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 226/457] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:32.708371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bb781-c399-4391-881b-0d681f648fde None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.708648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bb781-c399-4391-881b-0d681f648fde None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.708744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:32.708900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bb781-c399-4391-881b-0d681f648fde None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.710482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 232/458] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:32.715618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a942fd-7049-41e1-b99e-7a7a6f9de4c8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.716141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a942fd-7049-41e1-b99e-7a7a6f9de4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.716635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a942fd-7049-41e1-b99e-7a7a6f9de4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.744335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-38a942fd-7049-41e1-b99e-7a7a6f9de4c8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:32.745795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-38a942fd-7049-41e1-b99e-7a7a6f9de4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:32.834865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 233/459] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:32.834865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c27cd-db87-45e6-958f-15c824e16a38 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.834865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c27cd-db87-45e6-958f-15c824e16a38 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.834865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c27cd-db87-45e6-958f-15c824e16a38 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.834865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 234/460] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:32.842790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.842790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.843115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.870058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:32.871385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:32.874278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0652f23-b820-4677-a0ad-242a44df213d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:32.911738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:32.912870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:32.913343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:32.913807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:32.924869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:32.927414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:32.944990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 235/461] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:32.945857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:32.947094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:32.947962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:32.948815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:32.948815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:32.948815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:32.948815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:32.955504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 227/462] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:32.957184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:32.980572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.010728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.066083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5278538-db1f-426c-aa98-9af08d063876 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.083719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7cefcf-9cee-436b-bbfb-8b4ca7ad6a11 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.103421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.104085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.104378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.104653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.105382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:33.105690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.107088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.118610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.119428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.119713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.119988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.128978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 228/463] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:33.128978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.130493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.131980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-2784bf3f-cac3-4eda-b7a9-fc5f82e9a5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:33.152666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:33.153530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:33.154368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:33.154368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:33.154368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:33.154368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:33.155042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 236/464] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:33.156322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99741c3b-8bab-4de5-92b2-52d7feaed691 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.156611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99741c3b-8bab-4de5-92b2-52d7feaed691 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.156888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99741c3b-8bab-4de5-92b2-52d7feaed691 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.158142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 237/465] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:33.159028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99741c3b-8bab-4de5-92b2-52d7feaed691 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.215025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.240006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01680aeb-c47d-4a03-840e-b49e1abbea72 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.259469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.273957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99741c3b-8bab-4de5-92b2-52d7feaed691 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.275686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.276489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.276772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.277049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.288210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.289707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:33.308253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:33.309121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:33.310648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:33.310648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:33.310648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:33.310648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:33.310648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 229/466] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:33.310648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.316950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabfcfa-39d0-4da8-964d-20e386d1310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.317630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabfcfa-39d0-4da8-964d-20e386d1310d None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.317919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabfcfa-39d0-4da8-964d-20e386d1310d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.318200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabfcfa-39d0-4da8-964d-20e386d1310d None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.318951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabfcfa-39d0-4da8-964d-20e386d1310d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.320835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabfcfa-39d0-4da8-964d-20e386d1310d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.354882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 238/467] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:33 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:33.354882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.354882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.354882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.369885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:33.371243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:33.450047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1581e-23fe-457a-9cd2-ec87c3574885 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.471463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 239/468] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:33.471463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.501259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.515098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.516164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.516164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.516569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.517379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:33.517751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.520018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.536372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 230/469] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:33.537493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.590389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.634286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfaa12-73ee-493f-a244-2e528d247a30 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.677590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.678258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.678474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97f39f-7e03-4de1-b394-f2e996b33d5d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.678943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.678943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.679621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:33.680189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.682514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.698111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 240/470] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:33.699199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.724462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.725264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.725543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.725815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.743490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.744970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:33.768043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:33.769029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:33.769906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:33.769906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:33.769906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:33.769906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:33.770117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 231/471] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:33.771403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.821359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.823620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720c399-c7c1-46fe-a49b-d10f940ab819 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.859093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.859766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.860078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.860329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.861036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.862529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:33.869874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:33.886714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649130b-ac7a-438a-9255-4a477de7f75b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:33.894858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 241/472] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:33 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:33.894858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:33.982217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24670549-fbc8-4d4d-a0db-024b9cf2a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:33.983169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24670549-fbc8-4d4d-a0db-024b9cf2a14c None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:33.983665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24670549-fbc8-4d4d-a0db-024b9cf2a14c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:33.984163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24670549-fbc8-4d4d-a0db-024b9cf2a14c None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:33.985101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24670549-fbc8-4d4d-a0db-024b9cf2a14c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:33.986793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24670549-fbc8-4d4d-a0db-024b9cf2a14c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:34.012333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 232/473] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:33 2022] POST /identity/v3/users => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:34.013852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a1070-94de-4693-929d-43fe6de666c2 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.014134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a1070-94de-4693-929d-43fe6de666c2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.014417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a1070-94de-4693-929d-43fe6de666c2 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.016252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 233/474] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:34.021840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.022036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.022268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9c4a-4257-47fb-9393-51eb11946ad0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:34.022399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.048473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:34.050039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:34.099672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:34.100366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.100654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.100934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.101687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:34.102000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:34.103475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:34.129215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 242/475] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:34.135393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:34.180042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 234/476] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:34.204762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:34.210455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:34.248357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:34.294122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbb888-396e-4ff5-8ef2-eaed1defff4b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:34.337550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721f71d-89ea-443c-8899-40742021bc28 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:34.340057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:34.340748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.341033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.341310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.342048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:34.343508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:34.369416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 243/477] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:34 2022] POST /identity/v3/users => generated 249 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:34.378696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:34.379450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.379734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.380009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.380789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:34.382172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:34.391484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.sql.core [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 13:04:34.391484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 13:04:34.391484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [parameters: {'id': 'ab507a772b5e42a1873c9226ce7f8f1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 13:04:34.391484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80507) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application conn.query(q) Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application result.read() Jun 07 13:04:34.409083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return ref.to_dict() Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 13:04:34.410599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application next(self.gen) Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application yield resource Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application next(self.gen) Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise self.value Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application session.commit() Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 13:04:34.411550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self.session.flush() Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application compat.raise_( Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 13:04:34.412407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 13:04:34.413286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application conn.query(q) Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application result.read() Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 13:04:34.414102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [parameters: {'id': 'ab507a772b5e42a1873c9226ce7f8f1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:34.415057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 07 13:04:34.415883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:34.416836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 235/478] 10.222.0.61 () {60 vars in 1131 bytes} [Tue Jun 7 13:04:34 2022] POST /identity/v3/roles => generated 144 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:34.418399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:34.560367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb770040-2ada-4084-87a9-1a2d5632c16f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:34.584611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:34.603688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c664e81-9ab1-483c-95f8-026a8bc356d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:34.604516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c664e81-9ab1-483c-95f8-026a8bc356d4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.604796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c664e81-9ab1-483c-95f8-026a8bc356d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.605080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c664e81-9ab1-483c-95f8-026a8bc356d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.616254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c664e81-9ab1-483c-95f8-026a8bc356d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-5c664e81-9ab1-483c-95f8-026a8bc356d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:34.624405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:34.625295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:34.626117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 244/479] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:04:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:34.631489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:34.730535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:34.740128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ac66-0e46-4019-a71d-2118c68c12f9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:34.777204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:34.777861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:34.778146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:34.778423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:34.779278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:34.779585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:34.780975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a179ff-8a52-4876-b69d-1e867e1fc7d9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:34.790602 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 236/480] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:36.060268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9d80-1ef7-45b5-adca-1684015537cf None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:36.060789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9d80-1ef7-45b5-adca-1684015537cf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:36.060789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9d80-1ef7-45b5-adca-1684015537cf None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:36.062412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 245/481] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:36.068123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139474d0-8154-4162-8be1-4aef70db836f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:36.068547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139474d0-8154-4162-8be1-4aef70db836f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:36.068718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139474d0-8154-4162-8be1-4aef70db836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:36.091845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-139474d0-8154-4162-8be1-4aef70db836f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:36.093037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-139474d0-8154-4162-8be1-4aef70db836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:36.189577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 237/482] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:36.224061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9d80-1ef7-45b5-adca-1684015537cf None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:36.364573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9d80-1ef7-45b5-adca-1684015537cf None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:36.409442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:36.410641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:36.411145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:36.411651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:36.423089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:36.431359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:36.432281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:36.433598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:36.436373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 246/483] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:36.443918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139474d0-8154-4162-8be1-4aef70db836f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:36.583147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:36.603376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139474d0-8154-4162-8be1-4aef70db836f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:36.654267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:36.654911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:36.655197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:36.655469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:36.656232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:36.656533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:36.657930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:36.667287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 238/484] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:36.672774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:36.703316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:36.779371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467e394-2e4a-4b92-bd06-b4a3a86bfeed None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:36.813889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:36.814675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:36.814985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:36.815259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:36.835228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:36.836696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-c8a73db8-78ef-4c02-8d7e-ae61d87caccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:36.857873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:36.858760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:36.859655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:36.859655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:36.859655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:36.859655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:36.863660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 247/485] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:36.868896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.055724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540986ec-f418-4759-8de1-5f98be74e0af None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.056090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540986ec-f418-4759-8de1-5f98be74e0af None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.056269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540986ec-f418-4759-8de1-5f98be74e0af None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.057882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 248/486] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:37.062942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8351-aa58-4fe2-9102-963c719126d4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.063511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8351-aa58-4fe2-9102-963c719126d4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.063511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b8351-aa58-4fe2-9102-963c719126d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.091868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4a7b8351-aa58-4fe2-9102-963c719126d4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:37.093118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4a7b8351-aa58-4fe2-9102-963c719126d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:37.126588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6fa4e-bfe9-4aca-aef7-dbe4655968cc None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.167676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.168623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.168910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.169187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.170177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:37.170890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:37.172288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c390fb3-ecfe-40de-8e54-9980b6b986ae None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:37.192769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 239/487] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:37.194090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494178ca-3f08-4508-a5a0-dcc5aaf9f677 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.194369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494178ca-3f08-4508-a5a0-dcc5aaf9f677 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.194650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494178ca-3f08-4508-a5a0-dcc5aaf9f677 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.195947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 240/488] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:37.196581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494178ca-3f08-4508-a5a0-dcc5aaf9f677 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.198727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 249/489] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:37.207424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:37.209607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.209902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.210230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.232350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:37.233527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:37.235604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:37.309713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 250/490] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 07 13:04:37.310490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.332683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494178ca-3f08-4508-a5a0-dcc5aaf9f677 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.369686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.370865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.371344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.371828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.379839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:37.381530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:37.399001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:37.399926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 13:04:37.400811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:37.403128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 241/491] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:37.404110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.438321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551758b-e43a-4934-8010-1feb70a8fcef None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.475871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.476725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.477014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.477292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.484984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:37.485624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:37.489704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:37.497700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:37.498590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 251/492] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:37.504935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.529791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3522f91-4fdb-4b33-a6ef-8136a194bfb4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.564887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fbf05-a0a2-43b7-ad89-11a847bf71d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.565690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fbf05-a0a2-43b7-ad89-11a847bf71d1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.566300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fbf05-a0a2-43b7-ad89-11a847bf71d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.566586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fbf05-a0a2-43b7-ad89-11a847bf71d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.576955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fbf05-a0a2-43b7-ad89-11a847bf71d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-636fbf05-a0a2-43b7-ad89-11a847bf71d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:37.585409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:37.586305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:37.587780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 242/493] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:37.592586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fcf61-b7a1-4fa0-8a33-809107d07931 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.592877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fcf61-b7a1-4fa0-8a33-809107d07931 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.593156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fcf61-b7a1-4fa0-8a33-809107d07931 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.594575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 243/494] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:37.595167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fcf61-b7a1-4fa0-8a33-809107d07931 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.614591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da4b27-4599-4380-9a92-44d9bbe34a2d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.648739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.649398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.649946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.650234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.651114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:37.651425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:37.652879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b217903-267d-4bc5-b7a3-475ae5e90fcf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:37.699818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fcf61-b7a1-4fa0-8a33-809107d07931 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.726017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:37.737634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.738318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.738604 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.738917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.739696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:37.740008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:37.741428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:37.750530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 244/495] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:37.751438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.813015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 252/496] 10.222.0.61 () {56 vars in 1137 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:37.813015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d3ef-b58b-4a73-b6ab-9af8e4438686 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.813015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d3ef-b58b-4a73-b6ab-9af8e4438686 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.813015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9d3ef-b58b-4a73-b6ab-9af8e4438686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.828225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-55c9d3ef-b58b-4a73-b6ab-9af8e4438686 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:37.829346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-55c9d3ef-b58b-4a73-b6ab-9af8e4438686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:37.856749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97864cfb-5988-49e9-84da-672f5f24ebf2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:37.877872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:37.890269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:37.891270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.891552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.891829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.892586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:37.892895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:37.894807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:37.895250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 253/497] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:37.896556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff5c7f-718a-4fae-b816-756fea75e8f8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:37.896836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff5c7f-718a-4fae-b816-756fea75e8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:37.897240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff5c7f-718a-4fae-b816-756fea75e8f8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:37.903353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 254/498] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:37.903353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff5c7f-718a-4fae-b816-756fea75e8f8 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.906193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 245/499] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:37.907073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:37.962935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.009427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff5c7f-718a-4fae-b816-756fea75e8f8 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.012438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de93a8-f64e-4216-ab4e-53fcd32696cd None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.042817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.043624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.043914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.044188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.048100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.048749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.049032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.049323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.054328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.055862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-1258ce57-c473-4796-be19-f2a87e559fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:38.073608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:38.075038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:38.075902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:38.075902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:38.075902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.075902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:38.075902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 255/500] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:38.076177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.076244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.076907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.093128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:38.094375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:38.118453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.119967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.124076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.162279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 256/501] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:38.163127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.172709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.223045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 246/502] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:38.223881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.266510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4b6af-697c-41de-9833-044556b24ce7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.299304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.300092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.300862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.301137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.310910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.312359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.319983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad45e8-8e53-41a9-9ee9-a87abb0348c8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:38.330001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:38.331048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:38.332123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:38.332123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:38.332123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.332123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:38.332123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 257/503] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:38.332123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.355381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.356776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.357715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.357998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.358267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.369201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:38.376995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:38.378031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:38.379614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:38.382974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 247/504] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:38.383990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.397088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.436215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bdce1-dabd-489f-8437-a365f486bf79 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.469829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.470530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.470814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.471114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.471829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:38.472136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.473486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.484907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 258/505] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:38.485674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.486922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df58736-adc2-4d1b-9468-c0b1b54a8111 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.521038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.521823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.522096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.522367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.532437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.533893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:38.550997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:38.551836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:38.552678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:38.552678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:38.552678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.552678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:38.552678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 248/506] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:38.552936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.570543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.589735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbe887-1e92-4ee8-ac3e-a44485e30ecd None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.611276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.623779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.624560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.624844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.625121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.635075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.636537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.651682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b0352-a4ec-4e26-a35f-a41c91c8f6c7 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:38.653364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:38.654226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:38.655151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:38.655151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:38.655151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:38.655151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:38.655151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 259/507] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:38.655407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.690585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.691552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.691834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.692109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.692833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:38.693223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.694511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.706075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 249/508] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:38.706976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.759493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4150c3-efbf-4717-bfb2-7e218ddd22bc None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.788732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.794159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.794821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.795135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.795445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.796199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:38.796506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.797909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.805256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8c82b-f283-4c7e-a642-dc101e5822fa None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.806931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 260/509] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:38.807707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.828985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:38.843106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.843906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.844182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.844452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.851825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.853247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:38.867984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:38.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:38.869782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 250/510] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:38.870350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.906930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9db5-b29f-4181-98c9-5c7b627453d5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:38.941770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:38.942420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:38.942703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:38.943008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:38.943728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:38.944034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:38.945407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:38.957512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 261/511] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:38.958303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:38.974102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee28e-af30-4444-a370-4b468618f5b1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.007889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.008553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.008832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.009108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.009838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:39.010137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.011499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.011611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.024412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 251/512] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:39.025226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.061361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a43fa-ac50-4c48-9851-1b9cde5934b7 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.066392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.095428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.096207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.096483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.096756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.104200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.105618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:39.120891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:39.121725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:39.122621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 262/513] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:04:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:39.123214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.123991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340c70b-846d-4695-80b7-cb1ec428fda0 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.158231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.159036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.159316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.160126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.174638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.176611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:39.194278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:39.195324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:39.196187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:39.196187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:39.196187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:39.196187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:39.196187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 252/514] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:39.196453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.221951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19faaf5-9060-452e-8de3-73d95ea04517 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.235468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.256829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.257490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.257773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.258048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.258918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:39.259230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.260660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.277293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 263/515] 10.222.0.61 () {56 vars in 1137 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:39.278414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.292754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.302202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b704473-f79e-4ae1-b0cd-ee1004a5c588 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.335081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.335740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.336025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.336307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.337004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.338435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.358558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 253/516] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:39 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:39.359483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.390004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c8c3a-b532-4f22-823f-f04dc73b8cdb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.438394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.439359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.439642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.439908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.449384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.451485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:39.476934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:39.477882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:39.479202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 264/517] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:39.479930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fc4de-3250-45cd-9f26-4bce3c9d3297 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.480005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.512100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.518203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.518203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.518601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.518782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.519802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:39.520115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.521572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.566681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 254/518] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:39.567572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.577344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.594420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c00132-4850-42e4-99d5-b1433a174ef3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.631104 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.631871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.632158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.632435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.633160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:39.633465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.634851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.647373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 265/519] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:39.647945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.692739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07794a01-c3b9-464a-8354-fa30a48c6d92 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.733682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.734514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.734624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.734894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.745092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.756143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571709-9051-49e1-ba37-cda2e053e2bc None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.798653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.799355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.799639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.799990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.800858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:39.801117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.802539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.804425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:39.805868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:39.814539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:39.822054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 266/520] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:39.822679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.918190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 255/521] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:39.919471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:39.929976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2894c4-d75b-482a-96a4-652eb831a6d8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:39.965249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:39.965891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:39.966177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:39.966454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:39.989943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.021921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3e753-e14a-4afa-9883-42edebd73985 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.029855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.041685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.043163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.058876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.059678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.059962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.060226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.067772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.069237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:40.085715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:40.086762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 256/522] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:40.087950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.155079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 267/523] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:40.156004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.197009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9d4b1-8597-47cc-95f9-2d7dc11bed46 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.211036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.236993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.238023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.238404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.238686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.239552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=2fae9cab0b834583a01d0b57c78257af) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:40.239861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.241276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.254850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 257/524] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/role_assignments?user.id=2fae9cab0b834583a01d0b57c78257af&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:40.254850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.281138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d43e4-316c-4d4d-90eb-eb5d07100522 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.292265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.335606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.336426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.336732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.337666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.354374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.355977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:40.375664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:40.376596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:40.377442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:40.377442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:40.377442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:40.377442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:40.382337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 268/525] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:40.383307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.385138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b3034-4a7f-4b0e-bba4-3080bfb35d88 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.440252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.441960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.442385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.442849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.443847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:40.444895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.446782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.469667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 258/526] 10.222.0.61 () {56 vars in 1139 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:40.470705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.500179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50fef4-0573-4ceb-bb59-03c43f96c0f5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.502726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.541280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.542079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.542364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.542638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.554511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.556102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.563494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:40.577169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:40.578486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:40.579430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:40.579430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:40.579430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:40.579430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:40.579430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 269/527] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:40.579835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.580338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae81c7-836f-4dd1-b176-e6120be5766d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.619125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.620220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.620751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.621242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.622185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:40.622683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.624455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.637749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 259/528] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:40.638905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.698050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d519142-be60-4cb7-919c-ccc7c32b4162 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.735260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.735916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.736201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.736539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.758929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e776afb-c3e9-40bb-8083-907f794b9183 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.762473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.809321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.810905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.811122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:40.812008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:40.812770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:40.813266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:40.815059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:40.815563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:40.817437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:40.833968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 260/529] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:40.835058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.859206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:40.931432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 270/530] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:40.932453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:40.971428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be202eb-32e8-42b8-860d-52d1cbc0a0d5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:40.991936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.024783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.026048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.026716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.027184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.037265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.039432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.043095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39208b7e-16fa-4fda-93dd-c6fd7911db9e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:41.058572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:41.059660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:41.060707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:41.063057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 261/531] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:41.064063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.087874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.088664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.089945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.090142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.103147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.105566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:41.125284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:41.126185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:41.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:41.129136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 271/532] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:41.130041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.155643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.207438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3f2a-483c-4e62-8a38-e22065c8d040 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.248358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.249091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.249531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.249653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.256650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50ab7a-a595-4192-a67b-e297dc4eb9b1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.263428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.264890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.277760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:41.287157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:41.288020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:41.288892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:41.288892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:41.288892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:41.288892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:41.292862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 262/533] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:41.293720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.303921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.304867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.305373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.306294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.307738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:41.308518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.310437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.328100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 272/534] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:41.329304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.417369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83231d-b801-47f5-bfe9-72c38b871f93 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.439966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.457292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.458211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.458733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.458876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb773f7-247f-4c75-84a2-0cfb77cafc0d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.459401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.460334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:41.460953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.463361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.484197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 263/535] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:41.485528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.505102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.506084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.506556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.507194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.508113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:41.508592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.510150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.526916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 273/536] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:41.528048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.568393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.620063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1ea64-edb8-4a0b-99ba-98df145d6535 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.644581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd3737d-78c3-469e-9c3e-5333b6f9b2be None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.657459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.658116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.658398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.659029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.660755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=27c81d5738744687a915940977f0f647) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:41.661045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.662443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.674382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 264/537] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/role_assignments?user.id=27c81d5738744687a915940977f0f647&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:41.676040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.694172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.695611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.695905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.696181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.697047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=e56bcda01a104f5795f962662cadcd83) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:41.697361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.698882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0fa6d9-fd1e-4988-99a4-e010113142fb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.710861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 274/538] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/role_assignments?user.id=e56bcda01a104f5795f962662cadcd83&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:41.710861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265fbbd-cd48-407f-a5b7-f665eb405dbe None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.710861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265fbbd-cd48-407f-a5b7-f665eb405dbe None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.710861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265fbbd-cd48-407f-a5b7-f665eb405dbe None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.713522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 275/539] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:41.715013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.715308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.715591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.749039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:41.750290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:41.771905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.899812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a36bed-3b8a-4b09-bc49-44f7c0cf7ea1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:41.910584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 276/540] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:41.935475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:41.949655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:41.949655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:41.949655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:41.949655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:41.956990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:41.959275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:41.977143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:41.977801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:41.978625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:41.979486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:41.981708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 265/541] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:41.987498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:42.129177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f3531-88de-4550-b8d6-af027d53c938 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:42.140094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:42.141693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe82e1-43b6-4228-9b80-bc3c52d325a7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:42.183399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:42.184087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.184371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.184650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.185460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:42.185793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:42.187261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:42.208543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:42.209348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.209631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.209907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.212821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 266/542] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:42.219154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:42.237803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:42.245872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:42.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:42.247575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 277/543] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:42.253089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:42.337190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde6165-61b3-4844-9346-8eeead7aa0a0 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:42.392932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:42.400411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:42.401090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.401373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.401648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.402494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=6370d72aa28c4cce952ca67ce5452cbf) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:42.402804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:42.404276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:42.406295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573ccd1-00f5-48f7-b243-b1f7b450fe71 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:42.418316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 267/544] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:42 2022] GET /identity/v3/role_assignments?user.id=6370d72aa28c4cce952ca67ce5452cbf&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:42.444765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:42.445584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.445967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.446346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.447236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:42.447638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:42.449103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49186cf0-48b6-48fa-a552-c18f5421693d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:42.459392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 278/545] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:42.464829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:42.524619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:42.625201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414bd91-ff9e-44c4-9a78-a434b49a08a6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:42.670020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:42.670744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.671054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.671330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.682595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:42.684160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:42.702962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded75be3-6227-461f-a03d-2f6ba364a693 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.703260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded75be3-6227-461f-a03d-2f6ba364a693 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.703539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded75be3-6227-461f-a03d-2f6ba364a693 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.705155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 279/546] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:42.711342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:42.712229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:42.713165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:42.713165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:42.713165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:42.713165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:42.713383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.713383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.713383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.715597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 268/547] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:42.721166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:42.732959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:42.734807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:42.844613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 280/548] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:42.855564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:42.884593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1a148-37aa-4d0c-944c-7c07b533d35e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:42.909101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:42.932721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:42.933307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:42.933589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:42.933860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:42.934589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:42.934916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:42.936286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:42.961359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 269/549] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:42.966572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.049442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:43.071012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e3dd3-4991-488b-9c41-47a3a48a504d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.107648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.108751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.109235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.109680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.121563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:43.129604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:43.131053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:43.132028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:43.134595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 281/550] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:43.142080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.201188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb092b0-09b0-49ca-aabb-732d19bd2598 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.254737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44c2fb-38a9-4a4c-ba78-22ed9f01de3a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.285417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.286220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.286506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.286784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.293187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.294526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.295789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.296280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.301150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:43.301688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:43.303313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:43.305734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:43.308526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:43.315855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:43.322480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 282/551] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:43.330034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.338218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:43.338944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:43.339833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:43.340957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 270/552] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:43.347570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.457821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca9cc0-63a6-49cf-860f-7d3285120b4d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.508146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.508829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.509116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.509395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.510249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:43.510560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:43.512029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:43.528866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aca3d9-3b21-4307-bc8f-f2570d895e2a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.553376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 271/553] 10.222.0.61 () {56 vars in 1135 bytes} [Tue Jun 7 13:04:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:43.558716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.574343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.575177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.575460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.575733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.591706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:43.593431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:43.623732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:43.624922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:43.625987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:43.625987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:43.625987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:43.625987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:43.627889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 283/554] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:43.633342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.651348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:43.697751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137eaaa-b718-490e-b7ce-0c8fcc6f5447 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.733803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.734456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.734739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.735046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.768068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:43.825592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e4649-d96c-4778-8820-ce95a4742380 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:43.846785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:43.848889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:43.897216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:43.897888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:43.898171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:43.898445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:43.899205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:43.899511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:43.900890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:43.928139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 284/555] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:43.929081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:43.972387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.054511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9b24c-f1e4-43ff-b45c-ea864f2b6c41 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.078049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 272/556] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:04:44.084117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.091191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.100357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.103417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.103712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.103998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] PATH_INFO: `/v3/users/glance` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.124444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.126167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:44.165282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:44.166958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:44.168140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 285/557] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:44.172334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.210340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16864c40-3cb2-496d-a069-1d6ac92c5d04 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.248801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.249967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.250432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.250957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.264073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.266326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:44.286268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:44.287637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:44.288643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:44.288643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:44.288643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:44.288643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:44.291345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 273/558] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:44.300869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.328420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.355686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6d05b-12a9-4252-91a4-2094cc7d59af None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.393595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.394324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.394615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.394934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.395792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:44.396105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.397694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.416352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 286/559] 10.222.0.61 () {56 vars in 1137 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:44.424422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.470241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.480186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da3c3d-c4c6-490e-869d-e1abdb7a46e5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.520011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edda89-8810-4609-845f-d69672507af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.520676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edda89-8810-4609-845f-d69672507af1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.520958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edda89-8810-4609-845f-d69672507af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.521230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edda89-8810-4609-845f-d69672507af1 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.521959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edda89-8810-4609-845f-d69672507af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:44.522266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edda89-8810-4609-845f-d69672507af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.523681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edda89-8810-4609-845f-d69672507af1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.539608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 274/560] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:44.539608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a25e8-1258-4e18-8ca0-a38df56ae943 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.539608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a25e8-1258-4e18-8ca0-a38df56ae943 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.539608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a25e8-1258-4e18-8ca0-a38df56ae943 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.539608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 275/561] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:44.540020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e92bf-db9a-4eb2-b07e-5373216dc448 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.540020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e92bf-db9a-4eb2-b07e-5373216dc448 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.540743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e92bf-db9a-4eb2-b07e-5373216dc448 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.541626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 276/562] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:44.542191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e92bf-db9a-4eb2-b07e-5373216dc448 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.555814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e213d8-3763-4299-b38a-f924e51af945 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.591794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.592506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.592787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.593066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.604267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.605830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.618543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7e268eba-8fb6-4734-bbd9-f86977667273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:44.624691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:44.625607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:44.626508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:44.626508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:44.626508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:44.626508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:44.626508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 287/563] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:44.627170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63223e08-c67d-4225-87e5-283a3236519e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.627394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63223e08-c67d-4225-87e5-283a3236519e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.627673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63223e08-c67d-4225-87e5-283a3236519e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.644152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-63223e08-c67d-4225-87e5-283a3236519e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:44.645349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-63223e08-c67d-4225-87e5-283a3236519e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:44.657975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e92bf-db9a-4eb2-b07e-5373216dc448 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.696431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.697569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.698055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.698633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.706019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.708357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.712495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 288/564] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:44.713612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.713904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.714180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:44.729888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:44.730876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:44.731788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:44.735829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 277/565] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:44.736834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.741792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:44.743022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:44.756923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.820366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 289/566] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:44.822088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.847995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebe338-5f64-4832-b1b1-4438eae558ac None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.876843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:44.885007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.885841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.886131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.886411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.887175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:44.887485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:44.888885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:44.905895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 278/567] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:44.905895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:44.945225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344d322-b047-4980-bef0-6dff22c6c2e9 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:44.983724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:44.984517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:44.984802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:44.985072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:44.998597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:45.006649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfca37-adbe-477e-9569-32cffd859e8b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:45.008583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:45.009649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:45.010666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:45.015159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 290/568] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:45.016182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.025497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.045361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.046167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.046554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.046976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.047873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:45.048289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.049823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.067519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 279/569] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:45.067519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.117280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.127665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcec24-8c9e-42c6-b0c9-a57eb28b9900 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.160479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.161344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.161697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.161974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.167479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c104e7e-b4b9-46e0-b00f-cbd490b4a0d7 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.171854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:45.180035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:45.181123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 291/570] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:45.182108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.204764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.205553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.205836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.206111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.212710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.214154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:45.227974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:45.228926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:45.229798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 280/571] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:45.230285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.256489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.290063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c0915-8937-4854-84a4-bae0e7b66da5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.324744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.325482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.325771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.326046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.326800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:45.327137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.328536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeaad8-c13c-43c1-8af9-0f4ff7d990bf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.329343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb1af7-b004-4476-8493-44230e9fab71 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.337057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 292/572] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:45.338384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cf06b-d158-4c8c-9d9e-c7b6ee6fdb28 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.338684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cf06b-d158-4c8c-9d9e-c7b6ee6fdb28 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.338993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74cf06b-d158-4c8c-9d9e-c7b6ee6fdb28 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.340218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 293/573] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:45.340965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.341092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cf06b-d158-4c8c-9d9e-c7b6ee6fdb28 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.369740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.370312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.370594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.370890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.371726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=67519cd46f644a03a35cf1151287752e) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:45.376239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.377733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.383170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 281/574] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/role_assignments?user.id=67519cd46f644a03a35cf1151287752e&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:45.383943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.469271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74cf06b-d158-4c8c-9d9e-c7b6ee6fdb28 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.498935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8567f3-bf69-45f0-baa0-2850f88ea262 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.506102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.518050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.518734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.519091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.519375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.520147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:45.520589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.522015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.536256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 294/575] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:45.536256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.547294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.547966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.548252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.548525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.549344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:45.549639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.551841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e1809-349f-421e-a02b-712436df7bf0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.564756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 282/576] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:45.566143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.566752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.567121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.585969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:45.587209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:45.615190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.652234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92173499-355a-416d-9213-c28067c3d1d0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.666863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 283/577] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:45.666863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.703740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.704539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.704828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.705103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.716973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.718438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:45.736000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:45.736973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:45.738430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:45.738430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:45.738430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:45.738430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:45.738430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 295/578] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:45.739465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.768344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea586c26-8d7f-40ef-a22d-7ab7b0fe7561 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.783057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.803953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.804752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.805036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.805310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.815620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.817357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:45.835278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:45.836387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:45.837463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:45.837463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:45.837463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:45.837463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:45.841042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 284/579] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:45.841871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.847007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d605823-f781-48ea-a7c9-8153d812a72e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.860408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:45.883843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72039-aa50-4b84-b667-099337445660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.884733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72039-aa50-4b84-b667-099337445660 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.885023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72039-aa50-4b84-b667-099337445660 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.885303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72039-aa50-4b84-b667-099337445660 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/2fae9cab0b834583a01d0b57c78257af/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:45.902115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee72039-aa50-4b84-b667-099337445660 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=2fae9cab0b834583a01d0b57c78257af, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:45.903806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee72039-aa50-4b84-b667-099337445660 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:45.919263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 296/580] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:45 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/2fae9cab0b834583a01d0b57c78257af/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:45.920339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72039-aa50-4b84-b667-099337445660 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:45.950317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc3b7-073d-46c6-8f87-21a97b482534 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:45.990219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:45.991070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:45.991358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:45.991685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.012624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:46.021305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:46.022342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:46.023300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 285/581] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:46.024028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.065837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.078483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72039-aa50-4b84-b667-099337445660 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.125844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.127185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.127706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.128182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.129804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:46.130547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.132343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.139663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.147382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 297/582] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:46.148462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.172441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3cb0a-453c-48e5-84d2-901fb8d27e29 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.214614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abaebe-26db-4755-9749-632204e11592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.216074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abaebe-26db-4755-9749-632204e11592 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.216780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abaebe-26db-4755-9749-632204e11592 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.217241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abaebe-26db-4755-9749-632204e11592 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.218308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abaebe-26db-4755-9749-632204e11592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:46.219126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abaebe-26db-4755-9749-632204e11592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.221111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abaebe-26db-4755-9749-632204e11592 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.237131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 286/583] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:46.238622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abaebe-26db-4755-9749-632204e11592 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.263413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a86424-a800-45c6-82f6-b50243c5329f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.299298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.300071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.300455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.300837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.301633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:46.302194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.303593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.312336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 298/584] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:46.313243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.330431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.363754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abaebe-26db-4755-9749-632204e11592 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.401506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.402409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.402763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.403210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.406242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.412087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.414858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:46.431434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:46.432448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 287/585] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:46.433946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.441634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9cbda-c069-40bf-b115-57954837f1e5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.477014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.478108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.478554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.479012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.487059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.489288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:46.504737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:46.505878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:46.506851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:46.509546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 299/586] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:46.510750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.547606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae2fff-b6cb-4a93-b746-33eb4af6ce31 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.583836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.584749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.585655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.585655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.591330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.596608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.598406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:46.617566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:46.618488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:46.619409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:46.619409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:46.619409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:46.619409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:46.619409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 288/587] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:46.620538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.621660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9945776-f9c3-4cca-8dd8-d21c9ac26735 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.659293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.661349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.661810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.662319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.663468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:46.663968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.665709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.673042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.688356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 300/588] 10.222.0.61 () {56 vars in 1137 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:46.689457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.727676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7ed97-2c04-47a4-9dd4-bca3674fe2ad None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.763722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.764490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.764936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.765176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.766666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:46.766666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.767957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.785969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 289/589] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:46.786894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.797890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a34751-01ec-4dca-8af2-2217285a1298 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.831268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.847737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.848591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.849192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.849476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.850490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:46.850802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.852939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.868663 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 301/590] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:46.869560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:46.895504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe8294-59f9-42d7-9570-f6ff8dfbc735 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:46.904803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:46.937100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:46.937951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:46.938233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:46.938561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:46.949466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:46.950959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:46.969683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:46.970777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:46.971794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:46.971794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:46.971794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:46.971794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:46.971794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 290/591] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:46.972661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.000430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd6dc9-15ea-47a5-8171-815c4916065b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.051559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.053109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.053550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.054445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.070014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.072295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.078565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405ff4c-5e1f-419a-874c-8db8748923f8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:47.095902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:47.096995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:47.098599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:47.098599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:47.098599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:47.098599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:47.100569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 302/592] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:47.101513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.123036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.124081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.124553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.124980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.136937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.138969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.152982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:47.162051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:47.163215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 13:04:47.164371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:47.171833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 291/593] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:47.173263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.199075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:47.238121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7d567-960d-45c0-ac82-cdc484fc87eb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.273886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.274557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.274853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.275153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.275885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:47.276188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.277621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.291308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 303/594] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:47.292309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.298483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4dc0ad-bf6e-41cb-b7d6-39b7f02ad6c8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.345385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.346171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.346531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.346929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.348423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:47.348799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.351116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.372848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 292/595] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:47.372848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.451702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf0d74-e8ba-4fd1-ad95-fb7c189c1c9f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.454929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:47.470453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:47.495309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013e0f7-c82c-4e3c-a8b0-b85ceaab65c2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.498413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.499386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.499892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.500372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.501411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:47.501914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.503705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.530338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 304/596] 10.222.0.61 () {56 vars in 1139 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:47.531043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.531852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.532136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.532248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.532424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.540638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.542144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:47.557244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:47.558186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:47.559096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:47.563283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 293/597] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:47.563976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.646660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a43da1-9ec5-4b69-b43c-c7be4ca30d00 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.667973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c048d1-a377-4f32-8534-f95fae2f997b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.689204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.690593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.691672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.691957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.701158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.702704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.704115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.705351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.705351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.705639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.713012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:47.719181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:47.719400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:47.720266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:47.721149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:47.721566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.725292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 305/598] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:47.726128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-7b21d80a-ea89-43d3-bda7-abf0c6d031c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:47.740586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:47.741510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:47.742395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:47.742395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:47.742395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:47.742395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:47.742395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 294/599] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:47.743480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ba637-0671-43f3-b9d1-9f1f28bfb6cd None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.743761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ba637-0671-43f3-b9d1-9f1f28bfb6cd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.744033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ba637-0671-43f3-b9d1-9f1f28bfb6cd None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.747097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 295/600] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:47.747097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ba637-0671-43f3-b9d1-9f1f28bfb6cd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.760075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:47.838077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933feeb-2328-4cd4-a91d-9476aefef0f0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.868804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ba637-0671-43f3-b9d1-9f1f28bfb6cd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:47.872502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.873166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.873475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.873755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.874548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:47.874881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.876320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.891522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 306/601] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:47.892338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:47.911456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:47.912201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.912482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.912753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.913541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:47.913852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:47.915320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb833cb-75d0-4ac8-bf98-21eaa518a3e7 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:47.930166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 296/602] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:47.931265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:47.931556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:47.931835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:47.949448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:47.950642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:47.989962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.002608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.012722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed00aab-9adf-44a7-9492-24538aa1ecc3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.027711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 297/603] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:48.028765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.061874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.063474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.063938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.064626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.066794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:48.067531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:48.069721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:48.082978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 307/604] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:48.084011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.143460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b8865-f452-46aa-bb1b-8e457dd122e1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.182373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.183294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.183578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.183857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/27c81d5738744687a915940977f0f647/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.190620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741943a-c062-4bd0-8196-11576c107b33 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.199517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=27c81d5738744687a915940977f0f647, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:48.200967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:48.222858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 298/605] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:48 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/27c81d5738744687a915940977f0f647/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:48.222858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.242594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.243508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.244052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.244337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/e56bcda01a104f5795f962662cadcd83/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.256584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.265307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=e56bcda01a104f5795f962662cadcd83, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:48.265962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.268073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:48.290380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 308/606] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:48 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/e56bcda01a104f5795f962662cadcd83/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:48.291276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.355329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2db20-a934-4f06-a4a7-31c6bc09aabd None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.406914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4f92a-17a4-4501-9e4a-c2c341d97d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.408215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f92a-17a4-4501-9e4a-c2c341d97d3e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.408510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f92a-17a4-4501-9e4a-c2c341d97d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.409354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f92a-17a4-4501-9e4a-c2c341d97d3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.426305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455eab9a-37dc-46a9-ae1a-c56982f6b9e1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.429153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4f92a-17a4-4501-9e4a-c2c341d97d3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-00e4f92a-17a4-4501-9e4a-c2c341d97d3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:48.437218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:48.438281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:48.439174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:48.443951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 299/607] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:48.445485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ab55b-f82e-43f3-8728-d4060362ae6d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.445776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ab55b-f82e-43f3-8728-d4060362ae6d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.446055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ab55b-f82e-43f3-8728-d4060362ae6d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.450853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 300/608] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:48.454897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ab55b-f82e-43f3-8728-d4060362ae6d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.495331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.496646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.497131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.497604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.511012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:48.512835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-8f1571c1-438c-4d4e-9729-454283cd9f06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:48.528474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:48.529426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:48.530387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 309/609] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:48.531376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.531698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.532996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.552086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:48.553280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:48.585119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.603594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ab55b-f82e-43f3-8728-d4060362ae6d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.616670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.643744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 310/610] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:48.644646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.658207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.662936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.665003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.665302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.666118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:48.667622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:48.669029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:48.684103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 301/611] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:48.684945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.799442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a622d7-d38f-4ffe-812e-6193824ed0df None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.800233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880cdaf-2fa8-4eed-b3b3-9960d2e37de2 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:48.834795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.835729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.836079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.836423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.847898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:48.848087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:48.848566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:48.848852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:48.849131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:48.849936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:48.850248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:48.851726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:48.856540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:48.858373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:48.860353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:48.866930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 302/612] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:48.867146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 311/613] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:48.867317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.867474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:48.887318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.923337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:48.985381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab014913-bcfc-48dd-aeb3-fd1a348130a9 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.000930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757461d6-2d62-443e-9b7e-555ef0a9ebf0 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.020609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.021288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.021576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.021856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.022607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:49.022939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.024534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.032895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 312/614] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:49.033779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.045721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.047158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.047980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.048784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.063020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.065459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:49.088764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:49.089705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:49.090609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:49.090609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:49.090609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:49.090609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:49.092164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 303/615] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:49.092964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.143423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22fd5d-3336-42a9-9739-35f0779099b2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.146441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:49.175715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:49.181140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.182800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.183266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.183750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/6370d72aa28c4cce952ca67ce5452cbf/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.201820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=6370d72aa28c4cce952ca67ce5452cbf, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.203571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.226971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893e5f6-59a1-4196-acfa-d7782896b7ef None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.227397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 313/616] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:49 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/6370d72aa28c4cce952ca67ce5452cbf/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:49.228751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.265509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.266822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.267202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.267590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.281397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.283451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:49.305024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:49.305991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:49.306919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:49.306919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:49.306919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:49.306919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:49.306919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 304/617] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:49.313273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.392527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21284a-4eac-429a-89f5-c5f76a7f00b2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.462565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.464365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.465010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.466622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.467826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:49.469224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.471589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.496704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 314/618] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:49.503107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.522406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:49.536482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b5f4d-fa4f-4920-a27c-7adab6c01fe6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.557584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:49.578320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.579124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.579472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.579766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.580555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:49.580931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.582488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.596561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 305/619] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:49.601573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.623816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95078c9-bc32-4605-8cb2-01b8a0cac8ef None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.666497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.667625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.668091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.668528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.681099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.684150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:49.709042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:49.710272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:04:49.711487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:49.723733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 315/620] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:49.731797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.775432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3807fd4-2f41-418d-8775-6fcd3faca4a3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.811619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.812415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.812693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.812967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.822637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:49.837488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:49.838446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 07 13:04:49.839440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:49.842993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 306/621] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:49.844177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:49.880444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:49.945332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:49.949489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da38722-3252-478a-b0b1-1fba263b6929 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:49.984790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:49.985458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:49.985740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:49.986013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:49.986923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:49.987236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:49.989217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:49.991327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba65be-f434-4c74-8032-6e2dc22723ea None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.006959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 307/622] 10.222.0.61 () {56 vars in 1137 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:50.012227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.059818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.061067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.061488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.061899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.062979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:50.063349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.064998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:50.108404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 316/623] 10.222.0.61 () {56 vars in 1135 bytes} [Tue Jun 7 13:04:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:50.109494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:50.115293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.121500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e975d-af1b-4dc1-bd7b-cf690fd20c58 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.160865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.161515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.161795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.162064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.241835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.243327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:50.344725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:50.350958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 308/624] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 07 13:04:50.356946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.384901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b8d55-77d8-4439-acd9-37e9b31a6c15 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.476125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33bf64-91e4-4d6b-a6ea-b8b8d5c949e4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.492384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:50.515710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.516501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.516784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.517057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.527225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.527371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.528096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.528384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.528663 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.528765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:50.549472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.551494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:50.560563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:50.561802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:50.562725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:50.562725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:50.562725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:50.562725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:50.572844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 309/625] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:50.581317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:50.595748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:50.598261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:50.600157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:50.600157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:50.600157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:50.600157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:50.600157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 317/626] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:50.604105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.663284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:50.754067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917367d-a655-4ace-9f56-c724b3151f7c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.798491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.799180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.799465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.799742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.801076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:50.801381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.803837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098b7ea-3780-41cc-9b27-58fb2b015eaa None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:50.809523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99fe9f-4251-4d69-9689-f35a090e58de None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.821312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 318/627] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:50.822778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d2ed-0d63-404d-8b1b-5babebd9122b None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.823098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d2ed-0d63-404d-8b1b-5babebd9122b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.823380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4d2ed-0d63-404d-8b1b-5babebd9122b None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.824938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 319/628] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:50.825743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4d2ed-0d63-404d-8b1b-5babebd9122b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.851447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.852378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.852660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.852934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.853933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:50.854656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.856199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692d42c-3b8c-48e0-83fb-acb923ba3655 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:50.870966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 310/629] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:50.873679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65746516-385f-4b13-9415-7b73fe00488e None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.874194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65746516-385f-4b13-9415-7b73fe00488e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.874712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65746516-385f-4b13-9415-7b73fe00488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.884451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:50.894879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-65746516-385f-4b13-9415-7b73fe00488e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:50.896121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-65746516-385f-4b13-9415-7b73fe00488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:50.945357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4d2ed-0d63-404d-8b1b-5babebd9122b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:50.963148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 311/630] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:50.964453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ec90d-ba3a-4557-a297-71385f3b05a1 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.964743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ec90d-ba3a-4557-a297-71385f3b05a1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.965019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ec90d-ba3a-4557-a297-71385f3b05a1 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.966422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 312/631] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:50.970981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ec90d-ba3a-4557-a297-71385f3b05a1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:50.986560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:50.987763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:50.988224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:50.988671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:50.996113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:50.997957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.007881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-c7984833-6528-4622-b079-251c16e609a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:51.014236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:51.015361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:51.016454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:51.020560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 320/632] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:51.021787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.022078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.022352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.039547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:51.040708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:51.078573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ec90d-ba3a-4557-a297-71385f3b05a1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.108374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 321/633] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:51.109215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.125652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88586685-f779-40d2-876e-8425b02971ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.126857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88586685-f779-40d2-876e-8425b02971ae None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.127326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88586685-f779-40d2-876e-8425b02971ae None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.127855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88586685-f779-40d2-876e-8425b02971ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.136141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88586685-f779-40d2-876e-8425b02971ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.137623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88586685-f779-40d2-876e-8425b02971ae None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.150500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-88586685-f779-40d2-876e-8425b02971ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:51.152744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:51.153629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:51.154519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 313/634] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:51.155036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88586685-f779-40d2-876e-8425b02971ae None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.228464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a182b-bac5-46ac-bb9e-1e83560f3d35 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.260082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.265690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.266493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.266603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88586685-f779-40d2-876e-8425b02971ae None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.266787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.267068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.280123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:51.288400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:51.289500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:51.290370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 322/635] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:51.291095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.307732 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.308621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.309058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.309467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.310391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:51.310884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.312462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.325320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 314/636] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:51.326314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.395477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.403250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3a2a0-de07-4d79-b3b3-c14c91180a45 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.440715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6507-3770-4828-8193-daed650986da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.441832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508edd76-9132-4ba4-bbc4-bc4f17a06aac None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.442721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6507-3770-4828-8193-daed650986da None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.443216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6507-3770-4828-8193-daed650986da None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.443655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c6507-3770-4828-8193-daed650986da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.455091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c6507-3770-4828-8193-daed650986da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-d17c6507-3770-4828-8193-daed650986da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:51.463652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:51.464590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:51.465512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 323/637] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:51.466663 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6507-3770-4828-8193-daed650986da None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.481016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.481607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.481891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.482169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.483008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:51.483325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.484794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.498589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 315/638] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:51.504427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.526915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.583383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c6507-3770-4828-8193-daed650986da None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.617749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.618408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.618692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.618794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47968bfc-6bc3-4ee1-84d0-42ca6f15cd85 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.619260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.619737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:51.620042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.621775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.630141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 324/639] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:51.631028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.651155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.659413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.660347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.660623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.660913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/67519cd46f644a03a35cf1151287752e/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.675736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=67519cd46f644a03a35cf1151287752e, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.677207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.690509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 316/640] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:04:51 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/67519cd46f644a03a35cf1151287752e/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:04:51.692136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.759275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aa515-12a6-4e65-b9fb-f40468d39013 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.799163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.799936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.800898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.801178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.801974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:51.802294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.804131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.809678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.822529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 325/641] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:51.823566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.854647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ddd02-7a50-4afc-919b-cb1f5dc9f990 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:51.894352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:51.895069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:51.895329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:51.895656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:51.896506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=2fae9cab0b834583a01d0b57c78257af) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:51.896812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:51.898277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:51.908467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 317/642] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/role_assignments?user.id=2fae9cab0b834583a01d0b57c78257af&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:51.908467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:51.924785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:51.958054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bddb-6118-4a9f-b0c8-d919dbf7aa8c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.008818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.009782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.010065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.010400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.027293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.029145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.046755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3619795-f2cb-463f-b2d8-f1c857c93101 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7d3c2208-87fe-481b-a5b6-10dc505c26d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:52.051014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:52.052222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:52.053354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:52.053354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:52.053354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:52.053354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:52.055617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 326/643] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:52.057262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15b3a-18b0-4bf2-8577-0b064eed417d None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.057771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15b3a-18b0-4bf2-8577-0b064eed417d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.058281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d15b3a-18b0-4bf2-8577-0b064eed417d None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.060258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 327/644] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:52.061306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d15b3a-18b0-4bf2-8577-0b064eed417d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.098637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.107727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.108536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.108818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.109089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.123050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.124526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-c1305f31-5505-4a4c-a5e4-45914c8ddaaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:52.142578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:52.143586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:52.144690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:52.144690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:52.144690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:52.144690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:52.144933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 318/645] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:52.146109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.146413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.146669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.171605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:52.172802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:52.196389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d15b3a-18b0-4bf2-8577-0b064eed417d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.234952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.244133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.244831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.245241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.245519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.246289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:52.246599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.248081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.264853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 319/646] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:52.265101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 328/647] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:52.265902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.270001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.394951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fcd1-c509-4203-98e5-1a4aca6a152e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.438086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea16d9f9-d8aa-4d0e-bc6e-a3a25efb7800 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.443807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.444699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.445163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.445643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.446557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:52.447060 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.448622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.454634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.463029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 329/648] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:52.464202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.483917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.484841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.485200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.485792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.496959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.498921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:52.515165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:52.516136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:04:52.517129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:52.520447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 320/649] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:52.521615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.527184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.603043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d735a7-cdb2-40af-8065-7d0e8ba30a86 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.631524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f660ef-06e0-4c4c-a1b9-7c6cdad013ed None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.652531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.653427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.653828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.654174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.669152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:52.669901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.670921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.671396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.671842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:52.678081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:52.679102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:52.679955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:52.686950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.686950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.687528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 330/650] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:52.689319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:52.700690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:52.701666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:52.702623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 321/651] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:52.703264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.736733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.810090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ab34d-0daa-4ce1-8782-afa251d255f1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.831453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcb19e-8a38-42e3-9f6c-6b776cdd82c0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:52.844453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.853735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.855167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.855662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.856145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.857698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:52.858240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.860409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.872905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:52.873717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:52.878117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:52.878117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:52.878117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:52.878117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:52.878117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:52.879510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 322/652] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:52.880974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.900692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 331/653] 10.222.0.61 () {56 vars in 1137 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:52.901833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:52.991461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:52.998697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52793661-76de-44b8-bc4b-cdaaa6cb4cd3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.021702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa54db-de28-4915-92f0-bccd6a6c04ba None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.035525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.036043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.036322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.036596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.037432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:53.037738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.039195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.056961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 323/654] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:53.057990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.069548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.071045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.071964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.072727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.087510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.090177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:53.114081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:53.115019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:53.115928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:53.115928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:53.115928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:53.115928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:53.121442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 332/655] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:53.121442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.143581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.188090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796e071-fa27-43cd-851c-67c1dc9f03f5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.229205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427893b3-e0f7-45b6-8800-5466bec2ebef None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.234939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.235866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.236357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.236848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.265643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.266290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.266567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.266852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.268041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.331033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.333725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.345385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.346857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.369799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.461661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 324/656] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:53.462578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.475717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 333/657] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:53.476864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.527619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.591957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c352c-931d-4637-bd10-eaf59e2b7a9a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.592554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657a2e6-8fbc-49a2-9e42-7257a6834473 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.615669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.632164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.632955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.633236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.633508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.640928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.642690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.643335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.643925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.644325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.645119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:53.645662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.645836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.647896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.666872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 325/658] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:53.667512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:53.668402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:53.669300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:53.669300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:53.669300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:53.669300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:53.670524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.673113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 334/659] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:53.673972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.788595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b642fb-bfce-40db-b7a0-fff98506291e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.820632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c709f-26ba-4b36-9b75-5e4ba3e57b7e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:53.825215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.826005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.826286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.826558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.834158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.835648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.838649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:53.853811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:53.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 07 13:04:53.855771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:53.859810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 335/660] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:53.860799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.867477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:53.868622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:53.869105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:53.869617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:53.884819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:53.887131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:53.894453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:53.912858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:53.913813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:53.914693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:53.914693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:53.914693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:53.914693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:53.920858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 326/661] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:53.920858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:53.994140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0f44-cab8-4667-a344-3ea197a196a6 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.031003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea481-e072-4407-977f-93ab826094c2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.043939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.044661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.044887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.045161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.045896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:54.046265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.047630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.059647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 336/662] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:54.060697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.073821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.074511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.074906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.075187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.076026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:54.076332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.077882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450dd0-426a-4fbc-a5e0-d680b204c967 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.101326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 327/663] 10.222.0.61 () {56 vars in 1139 bytes} [Tue Jun 7 13:04:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:54.101326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e6ff2-e40f-4e48-be9c-fb8c27db107b None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.101326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e6ff2-e40f-4e48-be9c-fb8c27db107b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.101326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e6ff2-e40f-4e48-be9c-fb8c27db107b None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.101326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 328/664] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:54.101756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e6ff2-e40f-4e48-be9c-fb8c27db107b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.128972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.166954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.177007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f11446-17dd-48a6-b7c2-cbc53016a563 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.213801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.214469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.214751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.215050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.215771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:54.216071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.217450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.221937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e6ff2-e40f-4e48-be9c-fb8c27db107b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.229895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 337/665] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:54.234732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.259402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.260193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.260477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.260749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.267693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.269828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-f61ad982-9ccd-4a24-a770-c5b9c8359ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:54.285932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:54.286873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:54.287800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:54.292092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 329/666] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:54.292812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.293099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.293375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.310766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:54.311960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:54.336043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470aff1-0499-47c1-8f3d-c9b754305187 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.371434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.372077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.372355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.372631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.381965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 330/667] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:54.382729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.393729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.411265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.444635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.446107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.497933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb554f6-e5bf-44f7-981b-92498c1a7889 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.531148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.531938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.532217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.532488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.538936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.540366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.542393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 338/668] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:54.543256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-43d81220-82e2-43c0-8bdf-5bc79475f72e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:54.553945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:54.554914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:54.555862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 331/669] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:54.556373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b29b7-0c26-450c-8d67-253761aaa041 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.556656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b29b7-0c26-450c-8d67-253761aaa041 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.556837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b29b7-0c26-450c-8d67-253761aaa041 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.558407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 332/670] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:54.559116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b29b7-0c26-450c-8d67-253761aaa041 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.600599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.631163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.648353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ed6d7-ec77-4dae-83ac-485e5ec7dcc3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.662034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b29b7-0c26-450c-8d67-253761aaa041 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.680629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.681276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.681559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.681834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.682623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:54.682957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.684383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.694867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.695647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.695951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.696223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.696343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 339/671] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:54.697079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.710125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:54.717819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:54.718774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:54.719676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 333/672] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:54.721038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.799350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb44092-7541-49f9-8572-d9f764d1ea72 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.812337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.818495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0f183-7127-461b-a0fd-58b20679646e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:54.834619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.835425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.835704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.835978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.846113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:54.847145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.848584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.851411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:54.852054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.852338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.852614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.853399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:54.853706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:54.855146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-011e2111-7b81-4cbe-9479-5f418ec891be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:54.865947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:54.867036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:54.872816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:54.872816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:54.872816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:54.872816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:54.872816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 340/673] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:54.873119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 334/674] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:54.873200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:54.873278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.873357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:54.873357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:54.886236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:54.887449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:54.953039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 341/675] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:54.954063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:54.977607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85101ea-d5b0-4eb2-9fe2-113af88dcd29 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.011565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.012217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.012556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.012847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.013677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=27c81d5738744687a915940977f0f647) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.013991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.015452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.021203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 335/676] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/role_assignments?user.id=27c81d5738744687a915940977f0f647&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.021968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.063765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.074060 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc37c77-ede5-48d8-97ef-844c14d5d374 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.088103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.115936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.116620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.116901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.117172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.117936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.118239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.119654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.133564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 342/677] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.133564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.143532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aba9c-cd51-45e4-be58-8fd6483c411f None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.183122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.183812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.184100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.184378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.185127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.185439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.186948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.200735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 336/678] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.201582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.257457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687940-920c-4aff-a37f-8ab9e9dc3c88 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.294953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.295611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.295896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.296174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.297009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=e56bcda01a104f5795f962662cadcd83) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.297322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.298759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.305329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 343/679] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/role_assignments?user.id=e56bcda01a104f5795f962662cadcd83&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.306121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.319117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb76898-a18c-4021-8051-2c29230309b3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.337798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.340368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.369335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.370364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.370724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.371200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.384384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.385978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:55.407099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:55.408070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:55.408969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:55.408969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:55.408969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:55.408969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:55.408969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 337/680] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:55.410519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.461284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52b40b-565b-41df-a26c-41058db414d7 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.502036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.503066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.503308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.503629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.516514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.518028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.528278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9d47e-b3de-428d-ba53-c99780654f24 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:55.539163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:55.540077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:55.540972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:55.540972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:55.540972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:55.540972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:55.540972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 344/681] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:55.541688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.573149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.573961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.574247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.574526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.583039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.584506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:55.600553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:55.601508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:55.602447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 338/682] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:55.602973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.630554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.640577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.669126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f658418-0cbf-415f-ba00-a7dc7c5ae7f9 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.707651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.708343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.708673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.708946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.709685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.709988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.711395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.725211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 345/683] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.726084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.737813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681348-1c38-4463-bd8e-aec3eedea277 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.791774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.793369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.793821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.794248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.795397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.795708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.797088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.812695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 339/684] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.813514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.856141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63a510-a26b-47df-91c4-b1f0fe3cbcea None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.890649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.896718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.897376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.897656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.897926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.898933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:55.899715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.901693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:55.920643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 346/685] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:55.920643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:55.933594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:55.952155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e023bd-f83b-44bd-be4e-c031af43a8e6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:55.993711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:55.995110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:55.995607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:55.996088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:55.996998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:55.998741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:56.022084 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 340/686] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:04:55 2022] POST /identity/v3/users => generated 254 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:04:56.023416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:56.029267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a2659-d107-440b-93b1-75009840685c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:56.068304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:56.069085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:56.069308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:56.069582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:56.079006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:56.080898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:56.097489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:56.098736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:56.100387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 347/687] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:56.105215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:56.149453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66db9e8-1fe3-44be-9d7a-93be186e9e43 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:56.165637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:56.192241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:56.193474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:56.193964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:56.194435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:56.195777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=6370d72aa28c4cce952ca67ce5452cbf) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:56.196420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:56.198438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:56.211404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 341/688] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:56 2022] GET /identity/v3/role_assignments?user.id=6370d72aa28c4cce952ca67ce5452cbf&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:56.224402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326f05b-0dad-4321-a7b4-2308acf00ed7 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:56.265831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:56.266508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:56.266789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:56.267087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:56.267937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:56.268245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:56.269739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:56.290276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 348/689] 10.222.0.61 () {56 vars in 1135 bytes} [Tue Jun 7 13:04:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:56.295934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:56.310283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:56.431389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d11c0-ca02-4ca8-a815-3e152ea9bad2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:56.468733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:56.469391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:56.469671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:56.469941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:56.545814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:56.547312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:56.558300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:56.655582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 342/690] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:04:56.661215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:56.821178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec3151-1961-4dae-830b-9802b7e4d73e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:56.842445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:56.877027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:56.877830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:56.878113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:56.878386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:56.891883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:56.893327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:56.915991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:56.916972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:56.917882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:56.917882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:56.917882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:56.917882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:56.917882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 349/691] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:04:56.922947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:57.071780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a0530-3f40-4002-ad7c-7be00d40c8ee None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:57.093283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:57.128992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:57.129660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:57.129937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:57.130212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:57.130961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:57.131268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:57.132639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:57.149947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 343/692] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:57.155776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:57.301982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef6736-ebc9-4d44-8308-16451d5eaa64 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:57.344602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:57.345372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:57.345660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:57.345935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:57.356481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:57.357965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:04:57.372660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:57.373628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:57.374540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 350/693] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:57.380421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:57.519878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f687-b757-45f6-9414-9c513285ba3f None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:57.569609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:57.570278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:57.570558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:57.570857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:57.571649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:57.571959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:57.573526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed5af5-e376-41e9-8478-7e2cd7e71145 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:57.586929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 344/694] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:57.592629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:57.638246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:57.747416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b7741-9327-428b-b523-e04a1378aa35 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:57.804692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:57.805376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:57.805661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:57.805935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:57.806797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=67519cd46f644a03a35cf1151287752e) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:57.807134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:57.808625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb9df1-086b-484c-9d02-0dae3e64b7ee None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:57.821096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 351/695] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:04:57 2022] GET /identity/v3/role_assignments?user.id=67519cd46f644a03a35cf1151287752e&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:57.892824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c441e39-8895-41d6-876c-88cad2264c61 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:57.893066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c441e39-8895-41d6-876c-88cad2264c61 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:57.893303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c441e39-8895-41d6-876c-88cad2264c61 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:57.894881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 345/696] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:57.900446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48221624-db2b-4f28-927e-0cef14b97d70 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:57.901026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48221624-db2b-4f28-927e-0cef14b97d70 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:57.901487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48221624-db2b-4f28-927e-0cef14b97d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:57.923557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-48221624-db2b-4f28-927e-0cef14b97d70 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:57.925140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-48221624-db2b-4f28-927e-0cef14b97d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:58.005901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 352/697] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:58.040170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c441e39-8895-41d6-876c-88cad2264c61 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:58.085590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:58.176909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c441e39-8895-41d6-876c-88cad2264c61 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:58.215344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:58.216537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.217060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.217531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.218428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:58.220039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:58.233503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21ec40-ecc9-46cf-ba52-a40998297317 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.233999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21ec40-ecc9-46cf-ba52-a40998297317 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.234098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21ec40-ecc9-46cf-ba52-a40998297317 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.237537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 353/698] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:58.239291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:58.240277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:04:58.241124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:58.241920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.241920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.241920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.244234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 346/699] 10.222.0.61 () {56 vars in 1129 bytes} [Tue Jun 7 13:04:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:58.250074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:58.270004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:58.271321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:58.290493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:58.365417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 354/700] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:58.366042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:58.371287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffd22f-16be-4e88-ab9b-a648179b8919 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:58.391790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:58.412454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:58.413122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.413450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.413734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.414447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:58.414756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:58.416243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:58.423799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 347/701] 10.222.0.61 () {56 vars in 1124 bytes} [Tue Jun 7 13:04:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:04:58.427752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:58.523315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f5e70-9a2d-4d11-b6e5-0cfd59e20030 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:58.535568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c883d-90dd-43d2-8627-1b285bb28b66 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:58.570655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:58.574032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.574553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.575301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.582717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:58.583703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.584208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.584681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.585599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:58.586104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:58.587826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:58.595477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 348/702] 10.222.0.61 () {56 vars in 1124 bytes} [Tue Jun 7 13:04:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:58.601965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:58.614534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:58.615572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:58.616895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 355/703] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:58.622153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:58.664060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:58.774095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228a62-5f88-4337-91e3-169754299120 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:58.822171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:58.823263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:58.823796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:58.824275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:58.825249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:58.825850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:58.827503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:58.841390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 349/704] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:58.842689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:58.855202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:58.968153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc2f35-8214-4758-9765-d374e0115ab1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.013516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.014318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.014599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.014908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.035295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.036698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9c5510ed-34da-4bfa-a929-3f49df661169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:04:59.058525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:59.059811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:59.061311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:04:59.061311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:04:59.061311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:04:59.061311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:59.061311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 356/705] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:59.065486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.142670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6b24-da3a-49b5-86a0-973c6ff6ab1d None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.143040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6b24-da3a-49b5-86a0-973c6ff6ab1d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.143260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6b24-da3a-49b5-86a0-973c6ff6ab1d None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.144673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 357/706] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:04:59.152547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435eb1f-2143-4da0-b4c6-27aabf2d1db8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.152835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435eb1f-2143-4da0-b4c6-27aabf2d1db8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.153108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435eb1f-2143-4da0-b4c6-27aabf2d1db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.189739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:59.199155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62308dc-4887-4df9-9491-a74a97f47c23 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.199875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2435eb1f-2143-4da0-b4c6-27aabf2d1db8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:59.201076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2435eb1f-2143-4da0-b4c6-27aabf2d1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:59.236179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.237065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.237449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.237905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.238759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:59.239194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.240662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62934a39-b88e-4efe-86ee-3d7e26b9583d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:59.253271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 350/707] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:59.254527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ea428-8f6e-4bc2-b03c-944b8a1da47f None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.254923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ea428-8f6e-4bc2-b03c-944b8a1da47f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.255267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ea428-8f6e-4bc2-b03c-944b8a1da47f None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.260576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 351/708] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:04:59.260576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ea428-8f6e-4bc2-b03c-944b8a1da47f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.289040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:59.315244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 358/709] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:59.317309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.317776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.318253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.344320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:04:59.345516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:04:59.407385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ea428-8f6e-4bc2-b03c-944b8a1da47f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.438857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 359/710] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:04:59.438857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.471929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.473500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.474052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.474541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.482674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.484374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:04:59.500734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:59.501690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:04:59.502642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:59.505464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 352/711] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:59.506607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.514108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:59.549047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb1804-6eab-48d5-aa2e-60f1add55d03 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.569617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:59.590319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.591273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.591619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.591940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.604277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:04:59.612803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:59.613976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:04:59.615036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:04:59.621260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 360/712] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:59.621260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.656562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95209d1b-b0c3-4b36-b8bc-7783a766f050 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.702671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.703811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.704241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.704661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] PATH_INFO: `/v3/services/network` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.705479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.708279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:04:59.724147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:04:59.725419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:04:59.726475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:04:59.729856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 353/713] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:04:59.730775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.744299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c244-c8bf-4036-8f3e-36a4a2d7d490 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.788356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.789503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.790330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.790714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.791984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:59.792397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.794361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:59.817221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 361/714] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:59.818188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.818683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:59.857498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:04:59.867124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12666d5-d567-423d-940d-b243b72b3008 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.911847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.912895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.913390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.913866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.914940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:59.915510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.917156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:04:59.926153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 354/715] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:04:59.927341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:04:59.945648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10350ca4-acc1-4d41-97b1-3ebdcdd9f056 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:04:59.984304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:04:59.985421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:04:59.985931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:04:59.986459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:04:59.987437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] RBAC: Adding query filter params (name=network) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:04:59.988515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:04:59.990505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.000216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 362/716] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:00.001380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:00.058208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed67fa-7aa4-4aea-b80d-f5942a9e007a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:00.093421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:00.102667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272acda-f161-4378-b41e-2177555c245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:00.103846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272acda-f161-4378-b41e-2177555c245d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.104154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272acda-f161-4378-b41e-2177555c245d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.104434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272acda-f161-4378-b41e-2177555c245d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.115681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:00.119798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587345a-1bd6-49f9-b162-226c1c42af0b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:00.158867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:00.160089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.160574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.161158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.172494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:00.174385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.193616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272acda-f161-4378-b41e-2177555c245d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:00.195106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272acda-f161-4378-b41e-2177555c245d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:00.195909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:00.196782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:00.197705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:00.197705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:00.197705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:00.197705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:00.200077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 363/717] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:00.201063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:00.323730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 355/718] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:05:00.325018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272acda-f161-4378-b41e-2177555c245d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:00.330743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc570cb-bf4f-45e4-ac91-6de3693fcb61 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:00.361314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:00.396309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:00.397000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.397544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.397927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.398746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] RBAC: Adding query filter params (type=network) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:00.399168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:00.400743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.417891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 364/719] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:00.423877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:00.449026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272acda-f161-4378-b41e-2177555c245d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:00.477699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:00.512078 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:00.513490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.514545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.515228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.516302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:00.516804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:00.518762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.548979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 356/720] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:00.554403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:00.705262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a5bcb-ffa3-436b-be5a-6fc191120515 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:00.771026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37479243-9241-4741-991d-26675797f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:00.771880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37479243-9241-4741-991d-26675797f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.772160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37479243-9241-4741-991d-26675797f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.772436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37479243-9241-4741-991d-26675797f2f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.794736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37479243-9241-4741-991d-26675797f2f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:00.796216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37479243-9241-4741-991d-26675797f2f5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.806573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:00.822761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc184048-1003-4a76-bfd5-91a86df3d7fe None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-37479243-9241-4741-991d-26675797f2f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:00.843359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:00.844324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:00.845237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:00.845237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:00.845237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:00.845237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:00.849538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 365/721] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:00.855528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37479243-9241-4741-991d-26675797f2f5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:00.883875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:00.885086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.885595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.886069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.903480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:00.906944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-60477607-7be4-487c-906b-2aa5f0854201 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:00.944076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:00.945096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:00.946039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:00.950808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 357/722] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:00.954223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cc014-ab0d-40ff-9ca0-55b32b122d46 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:00.954512 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cc014-ab0d-40ff-9ca0-55b32b122d46 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:00.954791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cc014-ab0d-40ff-9ca0-55b32b122d46 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:00.963110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 358/723] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:00 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:00.964265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cc014-ab0d-40ff-9ca0-55b32b122d46 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.000873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.040519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37479243-9241-4741-991d-26675797f2f5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.089187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.089867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.090156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.090439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.091210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:01.091534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.092925 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7deb9a-e134-4354-bc2e-2526d745c1d4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.106211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 366/724] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:01.111881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.112176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.112456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.130534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:01.132036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:01.142728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cc014-ab0d-40ff-9ca0-55b32b122d46 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.177178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.177962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.178249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.178531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.179395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:01.179705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.181259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d8551-d758-467e-a89e-1389892f2a27 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.206496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 359/725] 10.222.0.61 () {56 vars in 1149 bytes} [Tue Jun 7 13:05:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:01.206496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debe35d-591f-447d-afd6-7163d489c1b1 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.206496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debe35d-591f-447d-afd6-7163d489c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.206496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debe35d-591f-447d-afd6-7163d489c1b1 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.206496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 360/726] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:01.206496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debe35d-591f-447d-afd6-7163d489c1b1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.219368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 367/727] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:05:01.219368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.219977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.280030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.318845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debe35d-591f-447d-afd6-7163d489c1b1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.324594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dddf81f-372c-4a98-84b7-d5b34f3087bb None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.353124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.353918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.354205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.354483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.360766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.361242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.361448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.361703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.361982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.362632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-df1a0979-58b4-4777-b6d2-854773bb84d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:01.376610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:01.377530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:01.379355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 361/728] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:01.379872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.379872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.380156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.397468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:01.398661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:01.444739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.446216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.469220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.477834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 362/729] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:05:01.478585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.520099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.567423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 368/730] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:05:01.571444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.583060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd7edf-4eed-4ba3-934d-2e998097d9e7 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.618359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.619218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.623089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.623089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.631548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:01.641569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:01.642480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:01.643349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:01.649525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 363/731] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:01.649525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.683541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cfad5a-68ec-4a83-844d-a2c50bea2259 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.709875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.721354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.722014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.722290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.722566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.723375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:01.723682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.725107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.741872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 369/732] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:01.741872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.761910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:01.763165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650dbf1-2db5-4f03-a3bf-378901967c93 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.799058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.799718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.800001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.800280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.801515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.802900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.819208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 364/733] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:01 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:01.820027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.866315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c688c-7240-4428-8e74-a1ea9d3249c7 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.920296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:01.920811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:01.921105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:01.921432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:01.934219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:01.935995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:01.955868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:01.956907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:01.957832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:01.957832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:01.957832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:01.957832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:01.957832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 370/734] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:01.958566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:01.960945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02361c9-ec4c-4edf-ac2d-a87703734bc5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:01.980742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:02.005444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.006481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.007034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.007521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.008661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:02.009232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.010998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.020741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 365/735] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:02.021939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.034129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:02.077456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958637b9-6ad1-476a-9410-861c679dfe1b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.115902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.116565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.116848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.117123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.118067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=e56bcda01a104f5795f962662cadcd83) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:02.118379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.126863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.126863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 371/736] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:01 2022] GET /identity/v3/role_assignments?user.id=e56bcda01a104f5795f962662cadcd83&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:02.126863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.167159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015d6bf-0c6d-4ec3-ad5c-f50cfa6a10cb None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.268771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.270354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.270862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.271362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.272308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:02.272822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.274422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.305481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 366/737] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:02.310964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.318594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb22516-c7b3-409e-ab65-e72171ef278e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.351750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:02.357941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.358793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.359164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.359648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.372824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.374304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:02.397513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:02.398459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:02.399400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:02.399400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:02.399400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:02.399400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:02.399400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 372/738] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:02.405708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.424991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:02.541284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d06bba-57d6-4a12-97bd-b2c4d6b7c3ef None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.588135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d277f-d181-4cba-8a32-1627159291f3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.623060 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.644801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.644801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.644801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.644801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:02.644801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.644801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.646873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 373/739] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:02.649637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.650890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.652095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.653057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.656189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.672447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.673938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:02.708301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:02.709363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:02.710332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:02.716154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 367/740] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:02.717099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.739144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:02.830041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e38fbe-9df7-454b-a929-67d305497bdf None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.876712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.877515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.877801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.878081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.887760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.889705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:02.906415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:02.907468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:02.908911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 374/741] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:02.916458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:02.917642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:02.922296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08040aac-6af0-4b03-948f-c23fa1d56791 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:02.975753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:02.976421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:02.976700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:02.976973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:02.977764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:02.978074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:02.981641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e57af-a7cc-4641-9ab1-dbf5f9cbac53 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:02.999584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 368/742] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:03.000784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f07dea-0b43-47b3-95de-d097133679ee None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.001079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f07dea-0b43-47b3-95de-d097133679ee None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.001361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f07dea-0b43-47b3-95de-d097133679ee None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.004407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 369/743] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:03.005415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f07dea-0b43-47b3-95de-d097133679ee None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:03.077525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99c43f-7200-4cfd-9447-ba2543de6a22 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:03.079964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:03.114652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f07dea-0b43-47b3-95de-d097133679ee None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:03.116821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:03.117504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.117797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.118083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.118957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:03.119277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:03.120941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17539f4-ee2f-470e-a1f6-01604828809d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:03.139266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 375/744] 10.222.0.61 () {56 vars in 1135 bytes} [Tue Jun 7 13:05:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:03.140522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.140813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.141092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.155149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:03.155802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.156080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.156359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.157193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=7af822c04e9747888820eebdb19ffea1) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:03.157510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:03.158065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:03.158970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:03.159297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:03.165180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 370/745] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:03 2022] GET /identity/v3/role_assignments?user.id=7af822c04e9747888820eebdb19ffea1&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:03.166079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:03.193944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:03.240089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 376/746] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 07 13:05:03.280372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:03.280599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a1ac-7cc5-48aa-954a-ad233eccfe3a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:03.320136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:03.320806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.321090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.321364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.403695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:03.405153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:03.436259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:03.498122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:03.508936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc24f7-ab1d-417c-bb62-1de173b55b25 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:03.551337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 371/747] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 07 13:05:03.556188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:03.557891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:03.558931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.559233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.559518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.560261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:03.561642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:03.611524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 377/748] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:03 2022] POST /identity/v3/services => generated 240 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:03.683460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41983d-3caa-463c-b26e-d88057c3a390 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:03.711612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:03.731114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:03.732122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.732684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.732966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.743969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:03.745464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:03.764305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:03.765307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:03.766797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:03.766797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:03.766797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:03.766797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:03.766797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 372/749] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:03.771795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:03.917501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef29bb2-c728-496e-ad14-27c4baf08807 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:03.953111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:03.954122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:03.954637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:03.955156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:03.956095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:03.957138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:03.958721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:03.971773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 378/750] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:03.977261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:04.082487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1520ac-c84a-4698-894d-c314edba8e16 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:04.121005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:04.122215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.122713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.123228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.135481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:04.137322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:04.155260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:04.156177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:04.157080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:04.159688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 373/751] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:04.167919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:04.180960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:04.303253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0525498-d17b-4363-98ae-f103bcc14126 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:04.350340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:04.351416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.351918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.352395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.353484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:04.354022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:04.355702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a61e0-1936-44c1-96c5-82ec7fd1f4ec None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:04.373819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 379/752] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:04.379924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:04.438436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:04.532648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b79d1-185a-4f9c-8558-164fd99fa662 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:04.572488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:04.573070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.573358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.573639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.574476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=67519cd46f644a03a35cf1151287752e) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:04.574792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:04.576285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40639b2-60bc-42f5-971d-a4345101181a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:04.582493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 374/753] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:04 2022] GET /identity/v3/role_assignments?user.id=67519cd46f644a03a35cf1151287752e&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:04.668819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca40b-580f-409f-9491-e9d501d285aa None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.669368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca40b-580f-409f-9491-e9d501d285aa None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.669619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ca40b-580f-409f-9491-e9d501d285aa None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.671271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 380/754] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:04.677180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceaea2-c453-421e-a9e8-81558375abc4 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.677474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceaea2-c453-421e-a9e8-81558375abc4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.677760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceaea2-c453-421e-a9e8-81558375abc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.696788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b5ceaea2-c453-421e-a9e8-81558375abc4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:04.698058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b5ceaea2-c453-421e-a9e8-81558375abc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:04.773248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 375/755] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:04.803827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ca40b-580f-409f-9491-e9d501d285aa None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:04.884175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc2cb9-8050-4c41-8910-395e38a57ff9 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.884478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc2cb9-8050-4c41-8910-395e38a57ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.884658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc2cb9-8050-4c41-8910-395e38a57ff9 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.886743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 376/756] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:04.891840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:04.892309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:04.892583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:04.916379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:04.917580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:04.921356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:04.951218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:04.977000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ca40b-580f-409f-9491-e9d501d285aa None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.036992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.037802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.038090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.038366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.039043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:05.040428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:05.048467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 377/757] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:05.065526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:05.066638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:05:05.067787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:05.071165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 381/758] 10.222.0.61 () {56 vars in 1129 bytes} [Tue Jun 7 13:05:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:05.072836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.092892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.225018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6617f8-79a4-4d7b-ad74-4a176b711a59 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.261383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.262208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.262500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.262698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12e635-c61a-480d-a690-3740758c1392 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.262815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.266728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:05.278998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:05.290040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:05.291116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:05.292577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 382/759] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:05.298393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.348104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.348774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.349059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.349330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.350042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:05.350362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:05.351756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:05.362058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 378/760] 10.222.0.61 () {56 vars in 1124 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:05.367481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.416166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:05.426552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d07c2b-4160-483d-8d83-39ca9ff71025 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.482569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.484370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.486175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.486617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.487554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:05.489351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:05.490915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:05.508948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 383/761] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:05.514635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.599385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a019fb4-6d77-4e1c-ab6c-77ceb99ffaa1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.610346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:05.663835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.664505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.664780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.665054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.665767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:05.666067 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:05.668805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc295cd-004b-4a3a-99ac-5f3e12bc25f0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:05.678203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 379/762] 10.222.0.61 () {56 vars in 1124 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:05.680718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a215-0dcd-417e-8376-0992a9d0df8b None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.681007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a215-0dcd-417e-8376-0992a9d0df8b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.681274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83a215-0dcd-417e-8376-0992a9d0df8b None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.682686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 380/763] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:05.687979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a215-0dcd-417e-8376-0992a9d0df8b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.697689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7201aec-a306-4a13-a239-c7393d814635 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.744550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.746396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.746893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.747673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.762635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:05.764412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-af8fc972-732c-4427-a9c5-0adf35c4d426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:05.782739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:05.783691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:05.784597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:05.784597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:05.784597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:05.784597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:05.785716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:05.787500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 384/764] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:05.789813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.790582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.791086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.809212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:05.810706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:05.812463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83a215-0dcd-417e-8376-0992a9d0df8b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:05.855415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:05.856147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:05.856551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:05.857158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:05.857949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] RBAC: Adding query filter params (interface=public, service_id=4bab59443ed54436a9c0a0813e611614, region_id=RegionOne) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:05.858259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:05.860981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:05.874924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 381/765] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/endpoints?service_id=4bab59443ed54436a9c0a0813e611614&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:05.876061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.898976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 385/766] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:05.900243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:05.959882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:06.006700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefcebf6-1e66-4181-bbbe-7ae4589c2e22 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.031574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3c03b-6598-4d34-9614-0e86301ac2be None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.040267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.040937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.041225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.041503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.042242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:06.042551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.043992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.056994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 382/767] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:06.057641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.072764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:06.073969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.074740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.075065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.075317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.076018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.077384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.090965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 386/768] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:05 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:06.092077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.182194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953f36b-7e97-4075-bb7e-51d961f9a954 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.222513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.223362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.223648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.223927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.235888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:06.246024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:06.247052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:06.247928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 383/769] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:06.255640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.307806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce75119-d044-42dd-b8bc-acf61abe2299 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.310797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:06.375544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.376340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.376621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.376894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.380252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:06.396117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.397593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:06.416403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:06.417428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:06.418367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:06.421328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 387/770] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:06.424220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc31ce-26d4-4789-ab5c-30fb5ebd9668 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.429451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.468405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.468977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.469262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.469538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.470318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:06.470629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.474067 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.484606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 384/771] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:06.490952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.597750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcce086-625d-4801-a9df-bd7463040430 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.657390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.658140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.658369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.658647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.660286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:06.660398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.661780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.666922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed186c-61f6-4a6e-bd50-66474934e6f5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.680475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 388/772] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:06.685829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.697821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:06.707701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.709079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.709682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:06.710469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.711392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.724475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.726285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:06.745637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:06.746591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:06.747846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:06.747846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:06.747846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:06.747846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:06.749961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 385/773] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:06.756631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.833485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22c941-a4c8-4be3-ba88-d61e66298aba None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.886972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f3d0d-fd7b-4205-8a22-1c7012d159da None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:06.907867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.909498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.910556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.911042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.927054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.928841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.930008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:06.930694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.931016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.931299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.932296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:06.932611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:06.934449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3a976-dd66-4def-a784-469f81ca283d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:06.949981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 386/774] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:06.951338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b26f-1e6b-4f78-9cc2-0631f34cc53d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.951627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b26f-1e6b-4f78-9cc2-0631f34cc53d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.951904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b26f-1e6b-4f78-9cc2-0631f34cc53d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7a3f11b5-51ed-4b32-b28b-ae9ebc67d4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:06.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:06.953710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:06.954611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:06.954611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:06.954611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:06.954611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:06.954852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 387/775] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:06.954852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661b26f-1e6b-4f78-9cc2-0631f34cc53d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:06.960694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 389/776] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:06.962634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:06.963682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:06.963682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:06.986582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:06.989102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:07.009868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.080669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 390/777] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:07.082199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.087044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.091848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661b26f-1e6b-4f78-9cc2-0631f34cc53d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.139412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.140579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.141079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.141557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.154269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.156787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:07.176770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:07.177798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:07.178747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 388/778] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:05:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:07.179468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.223391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2123d-88c7-4bd0-9898-244a93109348 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.267338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.268005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.268296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.268575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.269854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:07.270163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.271581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.283236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 391/779] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:07.283710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.283905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.290567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb50bee-4628-4790-95b7-093326695ed0 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.327602 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.328456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.328725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.329034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] PATH_INFO: `/v3/services/network` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.329691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.331074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9dffbd9e-7de3-4f30-8574-e11be96fabff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:07.343697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:07.344897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:05:07.345915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:07.350970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 389/780] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:07.353516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157bda7-1650-4b63-a171-4e1f470e895e None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.353816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157bda7-1650-4b63-a171-4e1f470e895e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.354094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157bda7-1650-4b63-a171-4e1f470e895e None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.355228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.355479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 390/781] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:07.360387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157bda7-1650-4b63-a171-4e1f470e895e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.405343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37d9e8-a3ef-40e3-8d38-6f5e15e9a0cc None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.449290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.450252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.450697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.451016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.451869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:07.452179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.453648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.470222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157bda7-1650-4b63-a171-4e1f470e895e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.473348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 392/782] 10.222.0.61 () {56 vars in 1149 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:07.474174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.512300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.513200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.513547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.513886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.524158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.525619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-09c919bb-eb07-4afa-bb94-713adfd0ac3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:07.540098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:07.541001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:07.541920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 391/783] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:07.542909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.542909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.543089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.561727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:07.562959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:07.582789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.589448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b74b69-5da7-4058-80ed-a8f6023191b8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.613363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.632273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.632932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.633216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.633491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.634194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] RBAC: Adding query filter params (name=network) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:07.634495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.635886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.643269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 392/784] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:07.643269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.650680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 393/785] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:07.651544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.761050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ef357-5a92-47f4-8abc-30f47ee52f1f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.761878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a06b9-df7f-4b83-8b85-b2c7866958ed None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.796749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.797403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.797687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.797961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.798742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:07.799075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.799662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.800942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.801057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.801219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.801495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.812686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.819435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 394/786] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:07.819589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.819674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:07.837544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:07.838473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:07.839462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:07.839462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:07.839462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:07.839462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:07.839462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 393/787] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:07.839758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.849404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.873033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:07.937725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b69f8-83da-42d5-b8db-4c9afcffae61 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.948021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469badaa-e685-4c59-ab0c-4b566731a509 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:07.974627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.975613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.976075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.976556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:07.977447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] RBAC: Adding query filter params (type=network) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:07.977953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:07.979526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:07.983937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:07.984728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:07.985010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:07.985495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 395/788] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:07.986150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:07.990525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.001342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:08.009016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:08.009985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:08.010899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:08.015175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 394/789] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:08.015948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.093358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:08.104778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e872138-c970-4222-8090-5032839f1a38 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.116189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:08.129994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431523-80af-4f3c-a45b-077266d8563c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.144025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.144947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.145232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.145512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/e56bcda01a104f5795f962662cadcd83/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.147743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=e56bcda01a104f5795f962662cadcd83, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.149325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.170875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 396/790] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:07 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/e56bcda01a104f5795f962662cadcd83/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:05:08.170875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.172310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.173065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.173357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.173638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.174407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:08.174723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.176826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.190823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 395/791] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:08.192478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.318291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed48290-ada3-4b9f-89ff-e722991b3103 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.333413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a4213-1ec3-466c-ad88-8b72263374ee None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.365795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.366913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.367682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.368154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.369210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] RBAC: Adding query filter params (interface=public, service_id=162adcf1c1c445c194fd2587ce5fe311, region_id=RegionOne) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:08.369718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.371635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.375264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:08.383172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 397/792] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/endpoints?service_id=162adcf1c1c445c194fd2587ce5fe311&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:08.384931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.392589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.393357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.393713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.394061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.395451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:08.395837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.397926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.412888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 396/793] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:08.414197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.431701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:08.509158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85408427-748a-45ce-9d3c-2d5c0b3325d0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.551379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff2c64-2e79-4f92-b52c-24a16ff54073 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.555728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.556541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.556826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.557103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.566968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.569304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.587805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.588513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.588791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:08.588997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:08.589863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:08.591631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:08.592296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.592296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.592448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 398/794] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:08.592448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.595609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.602320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 397/795] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:08.603154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.683031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:08.725408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:08.736860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c02f7-6e94-459a-b70b-3c5c44f687d0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.748369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d7205-ba2a-4258-89bd-9b6e674f9ce2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.776579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.777400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.777746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.778020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.788420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:08.789609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.790102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.790435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.790819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.791856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:08.791979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.792500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:08.794292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bf2e7-2582-4309-a959-9d1c74142e27 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:08.808344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 398/796] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:08.810027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4348627-cffd-4c15-8428-55547971ebf4 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.810525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4348627-cffd-4c15-8428-55547971ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.811048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4348627-cffd-4c15-8428-55547971ebf4 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:08.817669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:08.818622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:08.820091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:08.820091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:08.820091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:08.820091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:08.820091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 399/798] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:08.820642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 399/798] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:08.822608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:08.823153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:08.823653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:08.825851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.842204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:08.844659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:08.942859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383943-6a09-41b8-86d0-f59d09e44664 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:08.943081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 400/799] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:08.943081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:08.998385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.000055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.000700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.001501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/7af822c04e9747888820eebdb19ffea1/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.002671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:09.023782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=7af822c04e9747888820eebdb19ffea1, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.026160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.029536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:09.046659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 400/800] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:08 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/7af822c04e9747888820eebdb19ffea1/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:05:09.048273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:09.107674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a939d-518e-48e6-99b2-4e8c007f9054 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:09.190620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.192379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.192682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.195013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.195831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:09.196146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.200456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.225202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 401/801] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:09.230923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:09.245861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d043222-5fbe-4220-89eb-486565cf7eea None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:09.287537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.288401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.288690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.289147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.289818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.291235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:09.304116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:09.305098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:09.306158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 401/802] 10.222.0.61 () {56 vars in 1129 bytes} [Tue Jun 7 13:05:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:09.312956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:09.408659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb3ce4-1849-4a51-bab6-aaccb6bf3589 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:09.410738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:09.446994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:09.451467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.452272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.452553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.452826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.471751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.473255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:09.497162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:09.498182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:05:09.499193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:09.509815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 402/803] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:09.513721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:09.533847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d550a5f-8940-49f5-a252-af80f175e72e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:09.595988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.596651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.596934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.597209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.597955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:09.598256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.599650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.609228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 402/804] 10.222.0.61 () {56 vars in 1124 bytes} [Tue Jun 7 13:05:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:09.615351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:09.708063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8b420-0fbb-450b-b42d-5d907e438c56 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:09.774498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.775204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.775486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.775768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.776617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:09.776925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.778363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.796066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7518c-fc96-4e83-967b-13e774ac2aec None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:09.813256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 403/805] 10.222.0.61 () {56 vars in 1135 bytes} [Tue Jun 7 13:05:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:09.838528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:09.843708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:09.865189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:09.865914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:09.866181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:09.866500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:09.867238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:09.867541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:09.868903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:09.877900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:09.879703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 403/806] 10.222.0.61 () {56 vars in 1124 bytes} [Tue Jun 7 13:05:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:09.891747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:10.021135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46371e-81b5-4498-ab35-e5e00774c862 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:10.035686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac860a6-b9e0-4732-af97-285c6c921592 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:10.058190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:10.058858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.059148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.059431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.060368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:10.061663 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:10.087311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 404/807] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:10.132536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:10.133351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.133633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.133910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.155019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:10.156530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:10.185110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:10.186097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:10.187066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:10.187066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:10.187066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:10.187066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:10.187066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 404/808] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:10.192672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:10.266340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:10.338592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ef14-4c27-419f-86b0-80a7cd5fbb65 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:10.378164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:10.378968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.379377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.379662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.380403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:10.380713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:10.382110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:10.395432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 405/809] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:10.400521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:10.493472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:10.534708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00c082-ac64-4a2d-82d3-53afb58044d3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:10.588781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:10.589604 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.589891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.590170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.600496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:10.601940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-80fdfb09-bc5c-41d9-9b2a-25cbd599a13e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:10.619505 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:10.620462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:10.621963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 405/810] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:10.628395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:10.725766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496e030-39ea-45e1-b102-c49fe4b95d6d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.726415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496e030-39ea-45e1-b102-c49fe4b95d6d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.726415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496e030-39ea-45e1-b102-c49fe4b95d6d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.728047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 406/811] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:10.735003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.735544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.736036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.753739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:10.754977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:10.759454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44931798-b4b9-4092-afea-8bc81d47894d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:10.761252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:10.795786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:10.796466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.797031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.797031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.797787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:10.798092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:10.803850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:10.811756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 406/812] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:10.817791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:10.873458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 407/813] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:10.934335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:10.946140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fcb02-14fa-4cc2-81a1-c6ade17248d3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:10.981355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:10.982278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:10.982565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:10.982857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/67519cd46f644a03a35cf1151287752e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:10.984930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=67519cd46f644a03a35cf1151287752e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:10.986520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:11.002510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 407/814] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:10 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/67519cd46f644a03a35cf1151287752e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:05:11.047381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:11.120456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17420818-ca70-4423-ac28-cc6a008a0135 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:11.172493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddaa1d3-fd9b-4191-9bcd-66c3aabfc644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:11.173316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaa1d3-fd9b-4191-9bcd-66c3aabfc644 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.173601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaa1d3-fd9b-4191-9bcd-66c3aabfc644 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.173876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddaa1d3-fd9b-4191-9bcd-66c3aabfc644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.192781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddaa1d3-fd9b-4191-9bcd-66c3aabfc644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-eddaa1d3-fd9b-4191-9bcd-66c3aabfc644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:11.200573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:11.201532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:11.202392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 408/815] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:11.210851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:11.252601 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:11.297905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9925d-7e66-4cd9-90f3-151091110cd6 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.298224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9925d-7e66-4cd9-90f3-151091110cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.298451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9925d-7e66-4cd9-90f3-151091110cd6 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.299850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 409/816] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:11.307727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.307727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.307727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.324627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:11.326090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:11.338982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e901003-0fa3-4e6f-b4e8-f4d29083f56a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:11.380809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:11.381659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.381946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.382223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.383126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:11.383435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:11.384864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:11.395540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 408/817] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:11.400512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:11.410079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 410/818] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:11.431709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:11.503242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:11.514455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:11.546854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714cccb-42e8-4b50-bd7b-7375dbf43aad None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:11.574435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e324b22-3165-44a4-8117-dc35546c232d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:11.610525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:11.611354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.611679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.612020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] PATH_INFO: `/v3/services/network` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.612678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:11.614650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:11.627479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:11.628552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:11.629407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 411/819] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:11 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:11.632084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:11.632888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.633173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.633450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.643808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:11.649150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:11.650885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:11.669706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:11.670710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:11.671676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:11.671676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:11.671676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:11.671676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:11.673945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 409/820] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:11.679563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:11.811177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e2dd-e32a-48ff-a6db-ad29bcb64216 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:11.820121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e75603-1e23-4abb-b92d-3200e131de2b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:11.831171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:11.851289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:11.852299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.852793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.853350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.854268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:11.854582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:11.855988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0bf0c-f6a5-4c3a-a286-08b99545d0ed None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:11.861105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 412/821] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:11.862352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175c5d-695d-4639-bd1c-397128dea77a None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.862467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:11.862651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175c5d-695d-4639-bd1c-397128dea77a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.863714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.864005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.864286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.865081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:11.865392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:11.866769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddb2ab-27d1-4954-9284-8b4a4acba82b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:11.868079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175c5d-695d-4639-bd1c-397128dea77a None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.869729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 413/822] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:11.870165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:11.870493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef175c5d-695d-4639-bd1c-397128dea77a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:11.886928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 410/823] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:11.888661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:11.889115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:11.889458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:11.910565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:11.912468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:11.991100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef175c5d-695d-4639-bd1c-397128dea77a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.007440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 411/824] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:12.008626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.051807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.052483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.052768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.053095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.053809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:12.054161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.055579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.061403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 414/825] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:12.062136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.114363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:12.120825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c34d54-a015-459c-9243-5a0a5e2b6610 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.148268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:12.157050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.158027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.158615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.159044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.167914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.169491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.183167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aadaa6-dffd-4b0f-b436-d569ee7196e8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:12.184228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:12.185905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:12.188205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:12.196593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 412/826] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:12.196593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.221132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.221992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.222980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.223269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.234628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:12.243397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:12.245586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 415/827] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:12.247655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.303226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e629f-3088-4f2d-b8a1-166c1166d19f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.338631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.339245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.339536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.339814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.340558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.341927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.356012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:12.360074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e2729-b29b-45ab-b941-8fe6e77e0d95 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.367114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 413/828] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:12.367114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.459982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.461105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.461603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.462091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.466272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:12.467140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.468961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.474864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:12.509876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 416/829] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:12.516190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.576682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30aa51-65d4-4789-bd6e-3fbbb44afe4a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.618940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.618940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.618940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.618940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.619352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:12.619352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.619352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.630349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 414/830] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:12.634533 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.702307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86723786-d76a-4a93-9c22-8406df7b43c6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.751341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.752634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.753355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.754394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.760840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6b20e-d7cb-4eef-8150-c930c713d0fa None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.765866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:12.788046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:12.806546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:12.807393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:12.807683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:12.807964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:12.820797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.822257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:12.844758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:12.845698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:12.846603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:12.846603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:12.846603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:12.846603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:12.846603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 415/831] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:12.852650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:12.855043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:12.858282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:12.977683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8e11f-ed7e-4e66-89fc-c7c41f3ffcc8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:12.991501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 417/832] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:12.998866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.018564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.020019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.020456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.020970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.021962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:13.022425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.024110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.042706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 416/833] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:13.044071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.053019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.064615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.100467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf14f96-bc3d-4029-a2d7-e0d8d114f113 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.136286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.137125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.137394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.137674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.149610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.151157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.155342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a569305-bd0d-440b-a66d-88657b89fc15 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-450d0354-583e-4584-8d66-27eca79719d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:13.169350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:13.170299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 418/834] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88c29-43de-4178-87ec-7e5f7b3e8ca8 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88c29-43de-4178-87ec-7e5f7b3e8ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88c29-43de-4178-87ec-7e5f7b3e8ca8 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.176289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 419/835] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:13.178957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.179250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.179529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.198467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:13.199709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:13.207681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.208622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.208794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.209074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.216680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.218127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:13.233264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:13.234273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 417/836] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:13.235272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.280301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 420/837] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:13.281403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.327746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.335304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.361742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ea8b-c997-49dc-9d01-423047b56d2d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.397036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d6d7-4018-41e1-b8a5-5ec058120028 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.405720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.406746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.407059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.407342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.408069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:13.408379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.409763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.424682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 418/838] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:13.425550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.448524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.449471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.449755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.450029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.450894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:13.451206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.453048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.477425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 421/839] 10.222.0.61 () {56 vars in 1149 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:13.477425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.537272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a70f-6b5d-4161-87be-a1306522b2c0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.575313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.575602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.576405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.576768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.576976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.577628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.579088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.581924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.584482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa6119-1fee-4341-b4bf-23b9e17efb89 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:13.592480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:13.593488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:13.594358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 419/840] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:13.595277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.627045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.628149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.628434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.629259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.636555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.638011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:13.651692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:13.652760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 422/841] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:13.654328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.704238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441796f0-97e0-4bf5-ae37-4bf17b74f28c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.737521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.738670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.738979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.739261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.753163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58e276-4b30-44ea-8b71-6cab5d35d5b8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.785521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.786168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.786450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.786731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.787511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:13.787766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.789124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf982626-9191-4438-8d89-9ce7a1f86bb8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.793491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 423/842] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:13.794659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5bc1a-264e-47de-b211-ead7cd235dd9 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.794977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5bc1a-264e-47de-b211-ead7cd235dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.795254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5bc1a-264e-47de-b211-ead7cd235dd9 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.796624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 424/843] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:13.799883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5bc1a-264e-47de-b211-ead7cd235dd9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.804204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.806884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.808943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.820259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:13.903378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5bc1a-264e-47de-b211-ead7cd235dd9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:13.911933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 420/844] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:13.911933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:13.938022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:13.938671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.938990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.939264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.940045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:13.940350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:13.941758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe9d3d-0f0d-4943-8d97-e3611b0c9166 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:13.953509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 425/845] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:13.954678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:13.955127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:13.955403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:13.970464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:13.971671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:14.008481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.011364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5c6b-58a8-4ee5-ae57-f793e9ba5ff1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.036649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 426/846] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:14.037467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.045950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.046585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.046897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.047174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.047874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:14.048177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.049544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.058133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 421/847] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:14.059299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.063968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.167878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b699b-8986-4e58-aa94-bddb5f79ca6c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.168095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a25a7-de5a-4ff2-a891-61174c0e80d3 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.209476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.209701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.210141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.210314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.210410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.210558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.210709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.210812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.211582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=e56bcda01a104f5795f962662cadcd83) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:14.211854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.213317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.220007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 422/848] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/role_assignments?user.id=e56bcda01a104f5795f962662cadcd83&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:14.220813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.230224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.232541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:14.252745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:14.253783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:14.256259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:14.256259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:14.256259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:14.256259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:14.256259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 427/849] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:14.259977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.262922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.338894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.372543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c779e9-6c78-4fc1-a926-3b4ef9d68896 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.386631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55d7d2-103b-4a0d-aadc-695320597414 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.424713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.425559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.425845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.425950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.426179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.426548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.427064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.427343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.428075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:14.428378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.429766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.439171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:14.443609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 428/850] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:14.448781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:14.449202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:14.450110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:14.451018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 423/851] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:14.458531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.572811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.579029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d13b-c93f-4fc7-9dc1-180b3e25a897 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.587326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260c38d-517b-4f39-932f-fbb0e709885f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.607953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.624879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.625622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.625905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.626228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.627074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:14.627381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.629277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.629746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.631056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.631978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.632452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.638301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 424/852] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:14.641086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.642865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.644920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:14.663217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:14.664322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:14.665391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:14.668164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 429/853] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:14.673748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.779757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7209e591-8944-43d0-9a22-6dc8ebaa125e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.781678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e61ddf-a077-4fd0-a4cd-e947f3a21b0e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.817146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.818208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.818587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.819007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.819890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:14.820284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.821798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.832774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.834088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:14.834521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:14.834965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:14.840494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 430/854] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:14.841489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.848785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:14.850439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:14.875136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:14.876069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:14.876967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:14.876967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:14.876967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:14.876967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:14.876967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 425/855] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:14.887587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:14.891502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.899306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:14.960869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b3e59-20bf-40fd-98db-dfd7db47678c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:14.998676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:14.999629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:15.000011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:15.000382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:15.001311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=7af822c04e9747888820eebdb19ffea1) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:15.001714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:15.003280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:15.011366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff376d0-fdd5-4ea1-b0f9-7eb5059d333e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:15.015395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 431/856] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/role_assignments?user.id=7af822c04e9747888820eebdb19ffea1&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:15.051203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:15.051885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:15.052171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:15.052454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:15.053196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:15.053504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:15.054955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:15.069589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 426/857] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:15.075585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:15.188978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c306da6-88fd-48a7-9308-b9b2ab19fe67 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:15.192175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:15.227688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:15.228490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:15.228776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:15.229055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:15.236810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:15.238864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:15.254566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:15.255686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 07 13:05:15.256660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:15.260570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 432/858] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:15.262054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:15.327211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:15.368940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f0ae2-1a22-4cfb-bcb1-4893d46446a9 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:15.406603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:15.407306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:15.407591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:15.407872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:15.408770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:15.409080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:15.411109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:15.439449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 427/859] 10.222.0.61 () {56 vars in 1135 bytes} [Tue Jun 7 13:05:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:15.445209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:15.552940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c49b76-e93f-486f-a4cc-6c9127f04f1d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:15.591937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:15.592583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:15.592862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:15.593134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:15.625007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:15.663168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:15.664620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:15.774002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 433/860] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:15.779485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:15.882822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d7768f-1f5e-4717-8deb-019f5cd75e8f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:15.904078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:15.920624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:15.921485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:15.921765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:15.922084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:15.933114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:15.934593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:15.955050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:15.957749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:15.959888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:15.959888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:15.959888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:15.959888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:15.962875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 428/861] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:15.963977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:16.034430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:16.077125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61136984-2259-42f8-bc1f-b318e9c01d9f None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:16.115887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:16.116935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.117435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.117867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.118786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:16.119287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:16.120880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:16.138947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 434/862] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:16.141489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:16.262626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492c11f-6ac1-4203-ad5e-4977fbbb738d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:16.298535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:16.299301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.299586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.299858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.307085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:16.308533 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-af9376a8-b85a-4129-8a96-c5a48b5e1cd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:16.324007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:16.325232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:16.326514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 429/863] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:16.333713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:16.449133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5de0546-031e-44ed-8de2-0061a5855cd1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:16.457551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185f6e4-d8ce-491d-a8ca-d5cffbadf5fa None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.457882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185f6e4-d8ce-491d-a8ca-d5cffbadf5fa None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.458167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185f6e4-d8ce-491d-a8ca-d5cffbadf5fa None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.459834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 430/864] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:16.464655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.464952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.465237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.475354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:16.483703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:16.484922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:16.487163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:16.488459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.488748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.489165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.490026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:16.490335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:16.491857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:16.506918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 435/865] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:16.512079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:16.518481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:16.578999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 431/866] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:16.598396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:16.636070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e338336-157c-4876-92bc-c11fd82e8a36 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:16.674234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:16.675069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.675193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.675954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.676794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=67519cd46f644a03a35cf1151287752e) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:16.677102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:16.678524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17519ddd-f59c-4eeb-a9ba-2e6c4ff443db None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:16.691087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 436/867] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:16 2022] GET /identity/v3/role_assignments?user.id=67519cd46f644a03a35cf1151287752e&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:16.694498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17a7e7-3148-489f-a4e4-e9b533098ad2 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.695141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17a7e7-3148-489f-a4e4-e9b533098ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.696157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17a7e7-3148-489f-a4e4-e9b533098ad2 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.698984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 437/868] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:16.704740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.705080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.705299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.725757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:16.726977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:16.729270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4c8fd-a239-4983-a052-25f5b12c9615 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:16.769876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:16.771072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:16.771579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:16.772066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:16.773037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:16.774791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:16.778028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:16.793538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 432/869] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:16 2022] POST /identity/v3/services => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:05:16.821453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 438/870] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:16.845989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:16.879057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:16.970082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873eba48-881c-40a8-9508-e5cd559e4128 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:17.010784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:17.011632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.011916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.012190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.012846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:17.014205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:17.026842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:17.027892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:17.028766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 433/871] 10.222.0.61 () {56 vars in 1132 bytes} [Tue Jun 7 13:05:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:17.037017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:17.142965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57185bb-812f-4d08-b75c-0cf64cc0da09 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:17.179057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:17.179729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.180022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.180300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.181021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:17.181347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:17.182739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e0fa-7e5a-418d-8b0a-994378a11295 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:17.188579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 439/872] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:17.194666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:17.290621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:17.307211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a4b4-b28d-4711-8b42-4935ed71c3f2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:17.342269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:17.342962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.343248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.343524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.344240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:17.344545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:17.345927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0ddaf-8a56-4065-9f43-1a3ccea5208f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:17.351867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 434/873] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:17.547278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54a420-d1c4-4352-8a42-150ec9215030 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.547620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54a420-d1c4-4352-8a42-150ec9215030 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.547843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54a420-d1c4-4352-8a42-150ec9215030 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.549537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 440/874] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:17.555180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf257d-60a8-4209-889d-885476180047 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.555593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf257d-60a8-4209-889d-885476180047 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.555975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf257d-60a8-4209-889d-885476180047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.574726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5abf257d-60a8-4209-889d-885476180047 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:17.575987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5abf257d-60a8-4209-889d-885476180047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:17.648056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 435/875] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:17.671045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54a420-d1c4-4352-8a42-150ec9215030 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:17.719448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:17.786394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54a420-d1c4-4352-8a42-150ec9215030 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:17.821330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:17.822146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.822430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.822709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.835208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:17.843593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:17.845032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:17.846525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 441/876] 10.222.0.61 () {56 vars in 1120 bytes} [Tue Jun 7 13:05:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:17.850693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf257d-60a8-4209-889d-885476180047 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:17.934276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:17.958452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf257d-60a8-4209-889d-885476180047 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:17.993867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:17.994546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:17.994851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:17.995146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:17.995905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:17.996215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:17.997594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:18.007001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 436/877] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:05:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:18.015905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:18.116514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:18.120501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf396b2-c980-498b-8106-eceaca2506cf None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:18.156636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:18.157442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:18.157729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:18.158007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:18.169190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:18.170656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:18.188414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:18.189383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:18.190287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:18.190287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:18.190287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:18.190287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:18.190287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 442/878] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:18.199991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:18.303976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d03f17-a0a9-4a34-9ef7-78655bc8395b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:18.337479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:18.338183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:18.338466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:18.338738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:18.339481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:18.339782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:18.341147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:18.354410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 437/879] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:18.359786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:18.477887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a9f86-0da9-4ffd-a723-28cb58b48827 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:18.512797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:18.513620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:18.513911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:18.514185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:18.522144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:18.523618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:18.538965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:18.540990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:18.543315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 443/880] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:05:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:18.548243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:18.571278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:18.657640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba57e-11a1-49ec-a4b3-d4ee4d7851fb None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:18.694160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:18.694859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:18.695146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:18.695423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:18.696264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:18.696577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:18.698032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:18.716969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 438/881] 10.222.0.61 () {56 vars in 1149 bytes} [Tue Jun 7 13:05:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:18.723251 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:18.751517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:18.835860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf908-1c62-4694-b8c6-25e0a0323a08 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:18.874636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12580ab-629b-469c-88f1-1dc06673d9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:18.875322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12580ab-629b-469c-88f1-1dc06673d9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:18.875612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12580ab-629b-469c-88f1-1dc06673d9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:18.875887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12580ab-629b-469c-88f1-1dc06673d9d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:18.950063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12580ab-629b-469c-88f1-1dc06673d9d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:18.951566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12580ab-629b-469c-88f1-1dc06673d9d6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:18.963532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.053805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 444/882] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:18 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:19.059954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.094072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526e3dc-d37b-4d4f-a389-e9923f6a8871 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.094270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526e3dc-d37b-4d4f-a389-e9923f6a8871 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.094550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526e3dc-d37b-4d4f-a389-e9923f6a8871 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.096233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 445/883] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:19.102628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.103030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.103391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.120237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:19.121559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:19.127251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.169372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7a2fc-63ba-4969-8eb6-f7b9ca4dc1a2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.199043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 446/884] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:19.213292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.215186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.215599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.215995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.216397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.228453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.230051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4cbbaf24-35dc-4de1-8cd2-c9fb3fda9658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:19.249838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:19.252577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 439/885] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c860e-b9d2-4950-92e0-5bd02d330a06 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c860e-b9d2-4950-92e0-5bd02d330a06 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c860e-b9d2-4950-92e0-5bd02d330a06 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.256227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 440/886] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:19.259894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c860e-b9d2-4950-92e0-5bd02d330a06 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.262365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.320915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64c8fd-3399-4acb-bcfa-472500fb2506 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.355763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.356493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.356813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.357088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.357745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.359153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.370368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c860e-b9d2-4950-92e0-5bd02d330a06 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-ce1ed499-d13a-4213-9076-de275d79ff6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:19.371415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:19.372401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:19.373255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 447/887] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:19.374443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.374443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.374443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.391649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.391831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:19.393024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:19.416536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.417151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.417439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.417720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.418447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:19.418757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.421519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.434197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 441/888] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:19.435208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.473732 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 448/889] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:19.474803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.515553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.553756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badd489-a724-47ee-a432-629eb5ddb861 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.588113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.588535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c918569-211f-4b71-910d-7bb0500e1c89 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.588781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.589143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.589423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.590140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:19.590446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.591844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.596464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 442/890] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:19.597241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.631161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.632243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.632710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.633294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.633982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.641444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.643101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:19.656867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:19.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 449/891] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:19.658921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.704683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c52ffc-45f0-4140-ab24-36c675b15782 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.742628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.743452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.743740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.744008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.744654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.746025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:19.758506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:19.759840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:19.760721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 443/892] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:19.761537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.773820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.781101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78578615-f6e7-4dc3-af6e-8908a6ba6aae None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.818004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.818675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.818993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.819271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.819982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:19.820294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.821660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.827130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 450/893] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:19.827851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.880195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:19.883442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae08e2-d04b-4ec0-ad12-ee1e2df5ca8d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.928275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:19.928948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.929234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.929509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.930321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:19.930632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:19.932151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5f480-9a77-46bb-9fc6-10acf3ca24b1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:19.945908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 444/894] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:19.947168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174455-2af0-42c1-a52d-29f61742ebb7 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:19.947461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174455-2af0-42c1-a52d-29f61742ebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:19.947740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38174455-2af0-42c1-a52d-29f61742ebb7 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:19.949204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 445/895] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:19.951597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38174455-2af0-42c1-a52d-29f61742ebb7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:19.961663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01181b9e-1f60-479e-901a-50e79cf8d5a9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:19.999339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:20.000018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.000302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.000577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.001297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:20.001603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:20.003057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff1e39-336a-4f36-a6da-59cbc5fed423 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:20.012813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 451/896] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:20.012813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.012813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.012813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.026757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:20.027976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:20.047118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:20.080287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38174455-2af0-42c1-a52d-29f61742ebb7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:20.107126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 452/897] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:20.107856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:20.119134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:20.120058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.120058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.120318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.121283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] RBAC: Adding query filter params (role.id=379c934cc3ba4a03bf36ccefb53c4e0e, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=7af822c04e9747888820eebdb19ffea1) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:20.121594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:20.123254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:20.130883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 446/898] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:19 2022] GET /identity/v3/role_assignments?user.id=7af822c04e9747888820eebdb19ffea1&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=379c934cc3ba4a03bf36ccefb53c4e0e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:20.130883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:20.169441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:20.261235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a95e34-06f5-4d6f-ae6c-9557c1fd861e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:20.269085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd18e2b-0e7c-415f-a91e-c3f82c3817aa None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:20.305197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:20.305998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.306357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.306715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.307571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:20.307953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:20.309444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:20.315552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 453/899] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:20.316794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:20.317463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.317996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.318270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.326886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:20.326886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:20.327150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:20.338093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 447/900] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:20 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:20.360674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:20.455709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f041cc6-f810-4517-ac10-7595725c1318 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:20.500594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:20.501511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.501957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.502392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.503384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] RBAC: Adding query filter params (interface=public, service_id=a2b27e7c26d640c6b82fa3cc0b6189c9, region_id=RegionOne) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:20.503900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:20.505435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03afdf-41b7-4740-ace0-7ab88d63e8bf None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:20.516128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 454/901] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:20 2022] GET /identity/v3/endpoints?service_id=a2b27e7c26d640c6b82fa3cc0b6189c9&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:20.520618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:20.612792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:20.632000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844916c-d07b-47cb-8d7d-49c4c697b5b5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:20.667281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da611222-b055-45e0-8a96-fb3e5a37f364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:20.667992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da611222-b055-45e0-8a96-fb3e5a37f364 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:20.668278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da611222-b055-45e0-8a96-fb3e5a37f364 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:20.668551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da611222-b055-45e0-8a96-fb3e5a37f364 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:20.669231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da611222-b055-45e0-8a96-fb3e5a37f364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:20.670592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da611222-b055-45e0-8a96-fb3e5a37f364 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:20.681318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 448/902] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:20 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:05:22.123092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26390b5-c021-476d-bd93-66201359a105 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.123640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26390b5-c021-476d-bd93-66201359a105 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.123723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26390b5-c021-476d-bd93-66201359a105 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.125465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 455/903] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:22.131390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad18bd8-eb5b-4f91-85fc-3daf05028b92 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.131801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad18bd8-eb5b-4f91-85fc-3daf05028b92 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.131969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad18bd8-eb5b-4f91-85fc-3daf05028b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.149220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5ad18bd8-eb5b-4f91-85fc-3daf05028b92 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:22.150467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5ad18bd8-eb5b-4f91-85fc-3daf05028b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:22.223649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 449/904] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:22.242660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26390b5-c021-476d-bd93-66201359a105 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:22.324758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:22.349844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26390b5-c021-476d-bd93-66201359a105 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:22.389668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:22.390273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.390555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.390849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.391601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:22.392982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:22.408781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222b893-c543-45b5-8596-efbc7a085430 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.409107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222b893-c543-45b5-8596-efbc7a085430 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.409361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222b893-c543-45b5-8596-efbc7a085430 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.411476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 450/905] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:22.412262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 456/906] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:22 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:22.419438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221f415-64da-4a0e-a0b0-3e692d3e41dd None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.419438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221f415-64da-4a0e-a0b0-3e692d3e41dd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.419438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221f415-64da-4a0e-a0b0-3e692d3e41dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.447283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2221f415-64da-4a0e-a0b0-3e692d3e41dd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:22.448843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2221f415-64da-4a0e-a0b0-3e692d3e41dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:22.533571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 451/907] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:22.556647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:22.672996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752c053-3aef-4b19-828f-3de5edc7e1d1 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:22.709337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:22.710149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.710428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.710692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.713535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:22.719416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727f5b-6611-408f-856f-6a9baf0ea059 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.719712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727f5b-6611-408f-856f-6a9baf0ea059 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.720044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727f5b-6611-408f-856f-6a9baf0ea059 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.721953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 452/908] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:22.725043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:22.727358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.727653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.727940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.734280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:22.736338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:22.739366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:22.740216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:22.741894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 457/909] 10.222.0.61 () {56 vars in 1120 bytes} [Tue Jun 7 13:05:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:22.747384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:22.748072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:22.749048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:22.827924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 453/910] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:22.850604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:22.868486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0697a-d25a-4563-be69-9dd46267c3a4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:22.908778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:22.910632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:22.911395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:22.911905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:22.913545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:22.914066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:22.916650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:22.934409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 458/911] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:05:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:22.935656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:22.962199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245a918-29c1-432a-b619-6b2195513e9c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.002755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.003754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.004110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.004397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.005058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.005773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.006575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.008487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-6fbaa488-db62-4323-be6b-34e68e2acc5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:23.020573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:23.021588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:23.022460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:23.027269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 454/912] 10.222.0.61 () {56 vars in 1132 bytes} [Tue Jun 7 13:05:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:23.028321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0cae7-28ac-4d09-8419-5db5eb5938f2 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.028613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0cae7-28ac-4d09-8419-5db5eb5938f2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.028890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0cae7-28ac-4d09-8419-5db5eb5938f2 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.030351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 455/913] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:23.035623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0cae7-28ac-4d09-8419-5db5eb5938f2 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.056070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1f40e-7854-4b6d-9796-38db7a70f499 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.091037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.091929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.092315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.092647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.104172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.105616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9ec6c94f-a7bf-4af2-a033-a5222ad982f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:23.123802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:23.125302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:23.126225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:23.126225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:23.126225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:23.126225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:23.126225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 459/914] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:23.126519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.126773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.127075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.144119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:23.145339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:23.146382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0cae7-28ac-4d09-8419-5db5eb5938f2 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.181864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.182523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.182805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.183110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.183810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:23.184114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.185521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.195051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 456/915] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:23.195051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.220591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 460/916] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:23.231755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.258327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.287449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.299807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fa0b0-a0ef-4a00-b507-dfd8463e3d64 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.337595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.338317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.338694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.339154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.339780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:23.340087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.341454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.358921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 457/917] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:23.359134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53203a-1c4e-4cb9-9b95-3c073fbdd2ab None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.359248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.396258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.397013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.397303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.397579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.398288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:23.398596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.400003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.409542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 461/918] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:23.409542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.475809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f67bad-956b-4e99-b2ac-6f963020e781 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.500130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.508338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756508f-4187-4fe9-9a90-afa191de666b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.510743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.513812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.514698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.515020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.515351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.516002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.517363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:23.529473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:23.530634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:23.531542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 458/919] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:23.532338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.553289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.554727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.555181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.555848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.564821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.566220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:23.581351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:23.582852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 462/920] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:23.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.646311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebc949-235b-4b90-8faf-3b3c8d7a2af8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.683405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.684322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.684758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.685034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.685823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] RBAC: Adding query filter params (interface=public, service_id=531c0f69a41e40fdb42214d6a8fdcd79, region_id=RegionOne) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:23.686132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.687520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.689271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fb5a0-1f8e-4a49-a249-7476369e1129 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.693767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 459/921] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/endpoints?service_id=531c0f69a41e40fdb42214d6a8fdcd79&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:23.698696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.726651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.727292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.727543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.727821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.728528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:23.728836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.730207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.734961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 463/922] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:23.735727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.757427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.769307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:23.812568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ee5e6-6ee5-44b8-98ec-aef97a0d1e6a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.844184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e30d10-f6eb-4551-a998-206e559aa846 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:23.849408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.850077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.850364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.850638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.851691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:23.852001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.853625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.871772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 460/923] 10.222.0.61 () {56 vars in 1149 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:23.877078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:23.879371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:23.880037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:23.880321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:23.880599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:23.881278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:23.882644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:23.888782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 464/924] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:05:23.889625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:24.007969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b247c-b8f2-43d7-9963-748b8879866c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.022113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:24.036276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:24.047291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bc186-a810-489b-aabd-80fec8cebf0b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.062229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.062733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.063101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.063392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.064133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:24.064448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.065860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.072745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 461/925] 10.222.0.61 () {56 vars in 1114 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:24.078499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:24.100283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.101049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.101330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.101628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.112800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.115101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:24.135184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:24.136707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:24.137785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:24.137785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:24.137785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:24.137785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:24.137785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 465/926] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:24.142229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:24.222849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197ed53-be6a-4183-8b21-ceefe320092c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.254270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812d4be-5dfe-4b5b-892b-d070043a8f77 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.269799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.271161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.271452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.271728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.272487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.274703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.298995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.299668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.299956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.300238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.301542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:24.301973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.303392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.315597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 462/927] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:05:24.322503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 466/928] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:24.328357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:24.366143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:24.463787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed5387-1cb4-4f19-8260-d5847e353bd5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.505642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.506872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.507386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.507885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.516090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.517977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:24.535998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:24.536947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:24.537844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:24.540525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 463/929] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:24.549116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:24.553556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:24.662087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f45f28-28a5-48f5-8a56-0f26da086395 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.698155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.698890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.699182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.699462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.700263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:24.700574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.702026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47888-b933-48c1-98e0-d290a27b257d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.715933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 467/930] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:24.721699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:24.799530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238230f-2885-4c17-8683-397ba84f9c1e None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.799829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238230f-2885-4c17-8683-397ba84f9c1e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.800014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238230f-2885-4c17-8683-397ba84f9c1e None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.801714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 468/931] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:24.807157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.807440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.807717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.813832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:24.826815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:24.828518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:24.831151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c3c6d-9b1d-4cfb-a99d-b936ef50a72d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:24.867473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:24.868354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:24.868646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:24.868896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/7af822c04e9747888820eebdb19ffea1/roles/379c934cc3ba4a03bf36ccefb53c4e0e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:24.871223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=7af822c04e9747888820eebdb19ffea1, role_id=379c934cc3ba4a03bf36ccefb53c4e0e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:24.872675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:24.889966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 464/932] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:24 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/7af822c04e9747888820eebdb19ffea1/roles/379c934cc3ba4a03bf36ccefb53c4e0e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:05:24.907228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:24.919608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 469/933] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:24.938920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:25.051836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfed617-2ae3-4da3-a1c5-b473961451d3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:25.089159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:25.089963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:25.090244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:25.090514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:25.091193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:25.092552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:25.105371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:25.106393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:25.107291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 465/934] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:25.114461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:25.221344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5de49-8d04-4d66-b806-836ac92d69c5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:25.261145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:25.261815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:25.262099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:25.262373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:25.263133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:25.263440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:25.264825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:25.274936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 470/935] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:25.276630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:25.289135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:25.383824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022f061-7520-4c01-99ac-1ae9e46c1e52 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:25.418263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:25.422904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:25.422904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:25.422904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:25.422904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:25.422904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:25.422904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:25.436514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 466/936] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:25.436607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:25.523301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:25.539131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7924f-6759-4906-893e-360e196946aa None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:25.576778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:25.577454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:25.577791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:25.578076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:25.578918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] RBAC: Adding query filter params (interface=public, service_id=d7208dc5f33c4f16a1d104014235c756, region_id=RegionOne) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:25.579233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:25.580633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71dd0b-1b28-45fe-adb0-229d76a34780 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:25.590985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 471/937] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:25 2022] GET /identity/v3/endpoints?service_id=d7208dc5f33c4f16a1d104014235c756&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:25.592526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:25.665437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:25.697766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159bc0e-8562-4136-bdac-757f97b77d73 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:25.733683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909f72b-09ee-4dd0-92d0-163859f21655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:25.734351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f72b-09ee-4dd0-92d0-163859f21655 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:25.734638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f72b-09ee-4dd0-92d0-163859f21655 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:25.734982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909f72b-09ee-4dd0-92d0-163859f21655 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:25.735672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f72b-09ee-4dd0-92d0-163859f21655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:25.737164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909f72b-09ee-4dd0-92d0-163859f21655 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:25.747693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 467/938] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:25 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:05:26.351329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6513a-75bc-42eb-adc9-2adc295f4f61 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.352363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6513a-75bc-42eb-adc9-2adc295f4f61 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.352855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6513a-75bc-42eb-adc9-2adc295f4f61 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.354749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 472/939] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:26.360394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.361230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.361716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.379943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:26.381617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:26.466433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 468/940] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:26.489118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6513a-75bc-42eb-adc9-2adc295f4f61 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:26.600822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6513a-75bc-42eb-adc9-2adc295f4f61 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:26.624375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:26.636887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:26.637749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.638037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.638312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.638992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:26.640362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-8f2fc546-8089-48c4-987e-3042fa3445b5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:26.654042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:26.655081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:26.655950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 473/941] 10.222.0.61 () {56 vars in 1132 bytes} [Tue Jun 7 13:05:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:26.661667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:26.710652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6105e-f15f-4a23-8445-45799d4956f0 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.710999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6105e-f15f-4a23-8445-45799d4956f0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.711528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b6105e-f15f-4a23-8445-45799d4956f0 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.713348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 474/942] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:26.718352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.718636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.718944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.735613 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:26.736803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:26.739782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:26.770416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339ace-e47c-4f0f-a2a2-d7585a5d1486 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:26.807024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:26.807690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.807974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.808251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.808965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:26.809272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:26.810659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:26.814928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 475/943] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:26.816040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 469/944] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:26.821589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:26.834352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:26.934631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98824-5330-40f0-b2e4-8f8a5d15dc6b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:26.952510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee0989-b8e2-4757-ae30-e87efbe85acf None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:26.954512 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:26.979599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:26.980420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:26.980878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:26.981218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:26.982006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:26.982372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:26.983972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:26.990947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:26.996809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 476/945] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:26.998269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:27.000136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.001075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.001580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.002187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.003181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:27.004684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:27.006086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING keystone.server.flask.common [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 07 13:05:27.027579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.sql.core [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 13:05:27.027579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 13:05:27.027579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [parameters: {'id': 'ea092059644f4d2f90f103df583c907e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 13:05:27.027579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80506) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ea092059644f4d2f90f103df583c907e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ea092059644f4d2f90f103df583c907e. Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application conn.query(q) Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 13:05:27.054877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result.read() Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 07 13:05:27.055759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application next(self.gen) Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application yield resource Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application next(self.gen) Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise self.value Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application session.commit() Jun 07 13:05:27.057615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.session.flush() Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application compat.raise_( Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 13:05:27.058766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 13:05:27.059895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application conn.query(q) Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result.read() Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 13:05:27.060913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [parameters: {'id': 'ea092059644f4d2f90f103df583c907e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:27.061984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:27.063927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ea092059644f4d2f90f103df583c907e. Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.064840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 470/946] 10.222.0.61 () {60 vars in 1140 bytes} [Tue Jun 7 13:05:26 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:27.071167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:27.126913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284059a-ebdf-4809-8efc-ffe5ab261333 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:27.171489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.172799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.173479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.173905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.175101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:27.177377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:27.181796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d8e79-6b4e-496c-8b9d-13e465006a42 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:27.205160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 477/947] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:27.236278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.237394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.237826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.238291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.246664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:27.248538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-86e3f7ba-0182-4ff0-8a24-9e75f21d1cf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:27.264345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:27.265453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:27.266540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.268756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 471/948] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:27.280015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:27.319133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510572b-9557-47f3-b5d2-a8ec1d1a08ec None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.319521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510572b-9557-47f3-b5d2-a8ec1d1a08ec None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.319854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510572b-9557-47f3-b5d2-a8ec1d1a08ec None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.321386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 472/949] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:27.321656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:27.327051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.327342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.327618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.349701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:27.351443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:27.398355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:27.425722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fecfd-d575-43f6-9367-74dce6ac024a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:27.446821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 473/950] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:27.472917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:27.475974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.477490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.477783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.478062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.478894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:27.479163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:27.481404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:27.506939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 478/951] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:27.595899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6e254-b0e4-4726-b92d-4e1e09cd99ff None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:27.635573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.636400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.636683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.636960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.650178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:27.658885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:27.660094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:27.661426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 474/952] 10.222.0.61 () {56 vars in 1120 bytes} [Tue Jun 7 13:05:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:27.666403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:27.774971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:27.781256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7ea6-29f7-475d-9e6e-659a2df18df3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:27.818862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.819549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.819831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.820108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:27.820859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:27.821165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:27.822570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:27.831688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 479/953] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:05:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:27.837224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:27.876760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:27.951657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de9476-ef7b-4538-8070-bbd710e1fd74 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:27.987241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:27.988554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:27.988860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:27.989118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.001310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.002777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f1c04c17-f52c-4e22-af74-3dc7f0a74562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:28.021097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:28.022132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:28.023042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:28.023042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:28.023042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:28.023042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:28.023042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 475/954] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:28.028311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.079510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51563ff9-99ad-4efc-b3bd-47819dda94eb None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.079921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51563ff9-99ad-4efc-b3bd-47819dda94eb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.080093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51563ff9-99ad-4efc-b3bd-47819dda94eb None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.081682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 476/955] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:28.086597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.086923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.087209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.107405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:28.108613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:28.136040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0a049-e9c8-45c1-a276-734d5a63d4d2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.183164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.183938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.184289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.184654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.185562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:28.185936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.187484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.207091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 477/957] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:28.208713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 480/957] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:28.210416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.226584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:28.230983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.252867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:28.335564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4f9d-0a87-4012-a50e-30b71d5a1564 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.345245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96833e2d-e629-4483-afbe-31d3b6697fba None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.375051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.376249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.376687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.377108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.385397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.387211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.387710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.388760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.389107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.389457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.390355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.392298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:28.404509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:28.406014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 07 13:05:28.407241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:28.407918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:28.409099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:28.410520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:28.418359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 481/959] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:28.418359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.418798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 478/959] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:28.420112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.510569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:28.534190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:28.543254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3f533-c875-4ad2-b266-b373959b5aa5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.557782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cdd53e-323b-4840-8145-2998278c97b8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.583391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.584363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.584829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.585296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.586481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:28.587106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.588872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.600259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.600936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.601228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.601507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.602220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:28.602532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.603939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.614352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 479/961] 10.222.0.61 () {56 vars in 1149 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:28.614507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 482/961] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:28.615387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.615791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.716766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66324bb6-83cb-493f-91fe-ae07ad608dd9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.726316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d0ec0-6b63-4fdd-952f-e49973c5dd1c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.751299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.751934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.752218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.752494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.753227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:28.753565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.754991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.760243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:28.764800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 483/962] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:28.765821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:28.771345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.772221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.772685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.773179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.778987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:28.862092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.864199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.873231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad49d6-4e3f-4e04-be5c-5d0332dc3955 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:28.909072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:28.909738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:28.910063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:28.910352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:28.911137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:28.912519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:28.942635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 484/963] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:28.983607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 480/964] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:28.989777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.063951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:29.113728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f32a89-934a-495c-9b3f-6d7df21da84e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:29.153077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:29.154210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.154701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.155173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.168722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:29.170566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:29.188518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:29.189447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:29.190386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:29.190386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:29.190386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:29.190386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:29.192843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 485/965] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:29.199024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.236347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:29.310197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81c8ea-b1cd-4208-bf4f-758f78134dd8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:29.345567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:29.346230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.346512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.346786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.347587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:29.347892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:29.349260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:29.361238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 481/966] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:29.367218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.484058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e207-21a6-40f3-b0a6-1b3030594e73 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:29.507866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:29.519913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:29.521166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.521907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.522179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.529351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:29.530796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-679bf312-a7a3-45a4-bfdb-32d9523cff9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:29.546073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:29.547437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:29.548355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 486/967] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:29.558402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.566775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e509ef-4270-4c05-ace4-459cc31eb5be None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.567093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e509ef-4270-4c05-ace4-459cc31eb5be None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.567370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e509ef-4270-4c05-ace4-459cc31eb5be None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.569249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 487/968] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:29.574449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.575008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.575352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.598544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:29.599968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:29.674860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:29.687266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 488/969] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:29.690894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe99f4a-d885-4391-bd20-99c8d9952154 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:29.714920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.733960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:29.734616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.734927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.735204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.735992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:29.736296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:29.737717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:29.750737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 482/970] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:29.756695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.844285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f3f1f-bcc9-43c9-9772-00395d79bea3 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:29.847486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:29.864951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f694-98a9-490f-965f-af10b1ea017b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:29.888748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:29.889568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.889850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.890121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] PATH_INFO: `/v3/services/compute` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.890767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:29.892333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:29.905963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:29.907051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:29.907899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:29.911167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 489/971] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:29.911484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:29.912713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.913200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.913387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:29.913960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:29.915057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] RBAC: Adding query filter params (role.id=379c934cc3ba4a03bf36ccefb53c4e0e, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=7af822c04e9747888820eebdb19ffea1) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:29.915579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:29.918404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573d18-03d8-4ca5-a2fd-25b3896f0a71 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:29.926974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 483/972] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:29 2022] GET /identity/v3/role_assignments?user.id=7af822c04e9747888820eebdb19ffea1&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=379c934cc3ba4a03bf36ccefb53c4e0e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:29.996267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677766-1fe6-4956-943d-25fa6a048225 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:29.997376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677766-1fe6-4956-943d-25fa6a048225 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:29.998164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677766-1fe6-4956-943d-25fa6a048225 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.000248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 484/973] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:30.007028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:30.007670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:30.008172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.026374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:30.028013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:30.041085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:30.042450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70f40-c89e-439d-a0d6-c16857d5e1db None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:30.080874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:30.081560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:30.081886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:30.082162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.082920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:30.083226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:30.084712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:30.090520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 490/974] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:30.099967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:30.118942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 485/975] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:30.138593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:30.142007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:30.226575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94394f8-66e0-40e1-8d70-4dae27f9a583 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:30.259503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85379-889c-4d83-a183-c082b51bcc9e None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:30.264706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:30.265653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:30.266140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:30.266626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.267556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:30.268138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:30.269846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:30.275568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 491/976] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:30.297789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:30.298484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:30.298768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:30.299078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.299802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:30.301846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:30.317433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.sql.core [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 13:05:30.317433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 13:05:30.317433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [parameters: {'user_id': '7fea7cfacf424f468a2ca0e6baa40786', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 13:05:30.317433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80506) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application conn.query(q) Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result.read() Jun 07 13:05:30.325703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 13:05:30.326675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application next(self.gen) Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application yield resource Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application next(self.gen) Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.session.rollback() Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.force_reraise() Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise self.value Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application session.commit() Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 07 13:05:30.327946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._prepare_impl() Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.session.flush() Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._flush(objects) Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application compat.raise_( Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application flush_context.execute() Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rec.execute(self) Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 07 13:05:30.329054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 07 13:05:30.330213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result = self._query(query) Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application conn.query(q) Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application result.read() Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 07 13:05:30.331274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [parameters: {'user_id': '7fea7cfacf424f468a2ca0e6baa40786', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:30.332370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 07 13:05:30.333366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:30.334469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 486/977] 10.222.0.61 () {60 vars in 1131 bytes} [Tue Jun 7 13:05:30 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:30.341736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:30.460285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abefb1-c066-468b-be88-3c6039061219 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:30.467937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:30.497486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:30.498286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:30.498567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:30.498859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.507496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:30.508961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-489ff11d-8391-440f-9b08-aca88a4dc0ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:30.524139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:30.525189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:30.526416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 492/978] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:30.533838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:30.545948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:30.643475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aaa2e-045c-4834-b95e-784b39b74cac None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:30.677714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:30.678376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:30.678658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:30.678958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:30.679769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:30.680068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:30.681683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec82dd1-3ead-4074-a733-ea985821e337 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:30.721471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 487/979] 10.222.0.61 () {56 vars in 1101 bytes} [Tue Jun 7 13:05:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:31.401233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0a8ba-5fb1-453d-b295-ff196608f58a None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:31.401761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0a8ba-5fb1-453d-b295-ff196608f58a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:31.401871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0a8ba-5fb1-453d-b295-ff196608f58a None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:31.403227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 493/980] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:31.408506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:31.408810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:31.409089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:31.430278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:31.432264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:31.497010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:31.505694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 488/981] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:31.528769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0a8ba-5fb1-453d-b295-ff196608f58a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:31.635794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0a8ba-5fb1-453d-b295-ff196608f58a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:31.670374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:31.671143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:31.671430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:31.671707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:31.672363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:31.673729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:31.685935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:31.687246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:31.693338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 494/982] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:31.695921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:31.796954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac37c1-9a6a-4662-87f0-812470904e15 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:31.831665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:31.832906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:31.833229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:31.833542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:31.834296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:31.834642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:31.836102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:31.845877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 489/983] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:31.847443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:31.870407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:31.951540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19648-ebb6-476f-a7fa-dbc6da2b057a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:31.985559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:31.986245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:31.986622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:31.986937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:31.987680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:31.987994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:31.989454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.000118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 495/984] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:32.001956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:32.064917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:32.110094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada4379-af4e-4d4e-bdcf-1fa246e50622 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:32.145159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:32.145822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.146109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.146388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.147231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] RBAC: Adding query filter params (interface=admin, service_id=d7208dc5f33c4f16a1d104014235c756, region_id=RegionOne) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:32.147548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:32.148936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dfe90-4de2-402b-b014-91d969010cdd None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.158691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 490/985] 10.222.0.61 () {56 vars in 1249 bytes} [Tue Jun 7 13:05:32 2022] GET /identity/v3/endpoints?service_id=d7208dc5f33c4f16a1d104014235c756&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:32.160365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:32.249079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:32.266590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a394-2a88-42ca-9613-5461e4ebb87d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:32.299938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cff2-6bd6-42a1-9b04-64474af3212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:32.301129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cff2-6bd6-42a1-9b04-64474af3212a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.301414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cff2-6bd6-42a1-9b04-64474af3212a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.301753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cff2-6bd6-42a1-9b04-64474af3212a None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.302444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cff2-6bd6-42a1-9b04-64474af3212a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:32.303869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0cff2-6bd6-42a1-9b04-64474af3212a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.310907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 496/986] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:32 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:05:32.337688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77851973-5f49-4774-940b-9fd63a59634b None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.338041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77851973-5f49-4774-940b-9fd63a59634b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.338259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77851973-5f49-4774-940b-9fd63a59634b None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.339943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 491/987] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:32.345801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.346121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.346366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.364710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:32.365937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:32.440693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 497/988] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:32.464021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77851973-5f49-4774-940b-9fd63a59634b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:32.573470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77851973-5f49-4774-940b-9fd63a59634b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:32.598164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:32.609858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:32.610650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.610965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.611246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] PATH_INFO: `/v3/services/image` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.611898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:32.613854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-77f32752-411c-4b97-b437-b9333d301f0b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:32.625615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:32.626607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:32.628080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:32.631180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 492/989] 10.222.0.61 () {56 vars in 1105 bytes} [Tue Jun 7 13:05:32 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:32.632714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:32.651579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf719e8-a84d-4932-a183-211c60f28fc9 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.651897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf719e8-a84d-4932-a183-211c60f28fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.652133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf719e8-a84d-4932-a183-211c60f28fc9 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.653627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 493/990] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:32.658455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.658742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.659075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.676515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:32.677689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:32.715038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:32.740412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49983d-dcb7-461d-84e6-ffbe456d4115 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:32.749144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 494/991] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:32.771048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:32.775335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:32.775998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.776276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.776545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.777247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] RBAC: Adding query filter params (name=image) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:32.777551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:32.778932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.788696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 498/992] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:32.789723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:32.882641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79487146-adeb-4f21-86c6-9de0ea2eeb3a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:32.889907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c86eb-3044-4871-b21c-5d3efc78da10 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:32.901756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:32.921017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a9279-e1ee-445f-a069-1e73646028a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:32.921681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9279-e1ee-445f-a069-1e73646028a3 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.921969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9279-e1ee-445f-a069-1e73646028a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.922247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a9279-e1ee-445f-a069-1e73646028a3 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.922998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9279-e1ee-445f-a069-1e73646028a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:32.924353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a9279-e1ee-445f-a069-1e73646028a3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.928879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:32.929551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:32.929839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:32.930115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:32.930846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] RBAC: Adding query filter params (type=image) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:32.931165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:32.932593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7097004-aac1-4927-95af-eab169ad9e2b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:32.937565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 499/993] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:32.946954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 495/994] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:32 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:05:34.590342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdea131-94f7-4992-9530-5b46d34fe789 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:34.590918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdea131-94f7-4992-9530-5b46d34fe789 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:34.590918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdea131-94f7-4992-9530-5b46d34fe789 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:34.592435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 500/995] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:34.597519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:34.598380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:34.598574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:34.615393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:34.616572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:34.692136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 496/996] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:34.715185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdea131-94f7-4992-9530-5b46d34fe789 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:34.724651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:34.817611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdea131-94f7-4992-9530-5b46d34fe789 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:34.852589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:34.853426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:34.853711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:34.853988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:34.854659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:34.856105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:34.869255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:34.870332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 07 13:05:34.871603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:34.875265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 501/997] 10.222.0.61 () {56 vars in 1126 bytes} [Tue Jun 7 13:05:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:34.876709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:34.981268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78989b2-3a80-4a24-9bff-2f23b2c56e2e None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:34.983889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:35.017262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.017929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.018216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.018493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.019226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:35.020158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.021535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.032450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 497/998] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:35.032586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:35.103350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:35.135298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756a34c-ccd9-46af-a597-41fa718f0e9c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:35.168554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.169216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.169497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.169773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.170473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:35.170772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.172147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18f20e-a82b-4986-83ff-17738c5dfec8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.177130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 502/999] 10.222.0.61 () {56 vars in 1122 bytes} [Tue Jun 7 13:05:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:35.182503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:35.195883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8e761-050b-4f37-8c3c-aceb07359720 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.196377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8e761-050b-4f37-8c3c-aceb07359720 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.196795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8e761-050b-4f37-8c3c-aceb07359720 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.198249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 503/1000] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:35.205531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb990c92-998a-4432-b901-439d873f1d83 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.205828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb990c92-998a-4432-b901-439d873f1d83 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.206656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb990c92-998a-4432-b901-439d873f1d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.228989 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-fb990c92-998a-4432-b901-439d873f1d83 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:35.230907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fb990c92-998a-4432-b901-439d873f1d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:35.288358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369c0fa-3bfe-4801-86eb-78f9feccca42 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:35.298242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 504/1001] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:35.299753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618e681-4352-414c-a71c-3978135c40dd None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.300046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618e681-4352-414c-a71c-3978135c40dd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.300336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2618e681-4352-414c-a71c-3978135c40dd None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.302218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 505/1002] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:35.307416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.307705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.307986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.324658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:35.324899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.325556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.325844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:35.325951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.326139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.326897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.329171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.344350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 498/1003] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:35.345133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:35.380851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:35.392808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:35.411080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 506/1004] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:35 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:35.434818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:35.478644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff783951-b19f-4bc5-9d4d-c7974a4b768c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:35.518377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.519088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.519422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.519702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.520439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.521854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.537333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 499/1005] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:35 2022] POST /identity/v3/services => generated 242 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:35.548561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d98d01-6b5a-4f70-8525-d5c50f299dc5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:35.598076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.599560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.599560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.599560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.600136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.601536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:35.614943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:35.616031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:35.617072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 507/1006] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:35.625017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:35.723516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:35.742805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dd4b4-de38-4f13-81b0-e92cfb3e4405 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:35.779293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.779973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.780262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.780546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.781267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:35.781576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.782998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.789091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 500/1007] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:35.794794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:35.849475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:35.902732 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b371778-ae69-409b-8c6b-e555a912ddf2 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:35.938050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:35.938718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:35.939034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:35.939312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:35.940023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:35.940331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:35.941739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:35.951506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 508/1008] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:35.953054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:36.056826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ca40a-5a29-4833-a84f-27597057d7b5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:36.091137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:36.091804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:36.092089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:36.092366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:36.093168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] RBAC: Adding query filter params (interface=public, service_id=d72cfdde17574095bda9cbc4774538a3, region_id=RegionOne) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:36.093481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:36.094900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13238f-34aa-4a06-9a58-4d0b7a7a033b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:36.103140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 501/1009] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:35 2022] GET /identity/v3/endpoints?service_id=d72cfdde17574095bda9cbc4774538a3&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:36.108509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:36.210630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900d4a0-015a-45db-a29d-d6618f14050a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:36.224869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:36.245522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f646c-00b6-4bad-8714-701cf84a2881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:36.246272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f646c-00b6-4bad-8714-701cf84a2881 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:36.246622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f646c-00b6-4bad-8714-701cf84a2881 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:36.247000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f646c-00b6-4bad-8714-701cf84a2881 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:36.247756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f646c-00b6-4bad-8714-701cf84a2881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:36.249175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f646c-00b6-4bad-8714-701cf84a2881 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:36.262892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 509/1010] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:36 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:05:37.662357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9411637-c37c-465b-ab14-51e013fcf1a3 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:37.663136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9411637-c37c-465b-ab14-51e013fcf1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:37.663136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9411637-c37c-465b-ab14-51e013fcf1a3 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:37.664358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 502/1011] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:37.669434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a21f7-e2e8-4ba2-8caa-101f114e1905 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:37.669823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a21f7-e2e8-4ba2-8caa-101f114e1905 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:37.669980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a21f7-e2e8-4ba2-8caa-101f114e1905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:37.686934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1e7a21f7-e2e8-4ba2-8caa-101f114e1905 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:37.688032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1e7a21f7-e2e8-4ba2-8caa-101f114e1905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:37.754538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 510/1012] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:37 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:37.778147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9411637-c37c-465b-ab14-51e013fcf1a3 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:37.797352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:37.837530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d491a2-d760-4dbe-98bc-14b259d0970c None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:37.837854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d491a2-d760-4dbe-98bc-14b259d0970c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:37.838081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d491a2-d760-4dbe-98bc-14b259d0970c None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:37.839498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 511/1013] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:37.844481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:37.844776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:37.845083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:37.862114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:37.863500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:37.887198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9411637-c37c-465b-ab14-51e013fcf1a3 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:37.923443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:37.924233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:37.924523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:37.924807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:37.926101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:37.927579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:37.929200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 503/1014] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:37.941771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 512/1015] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:37 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:37.941916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:37.956372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:37.999661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:38.027927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:38.046802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42025548-8f43-40b0-8e4f-6d64d2c1755b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.069117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7394f-cce2-4351-8f85-4b678aae3b60 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.085533 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.086198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.086490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.086772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.087528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.088974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.106519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.107400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.107682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.107956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] PATH_INFO: `/v3/services/image` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.108608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.109956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.126860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 504/1016] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:05:37 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:38.127058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:38.128024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:38.129449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 513/1017] 10.222.0.61 () {56 vars in 1105 bytes} [Tue Jun 7 13:05:37 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:38.130723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:38.248040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1cb36e-61b0-4cd7-86b1-6d3acfbea0d1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.284876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.285593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.285876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.286149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.286890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] RBAC: Adding query filter params (name=image) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:38.287200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.288605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.290823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:38.299026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 505/1018] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:38.301474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:38.410374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf609e-a629-48ca-9d6d-c1ed986ed686 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.412092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:38.451589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.452256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.452536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.452808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.453510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] RBAC: Adding query filter params (type=image) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:38.453815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.455212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cd52-187b-4ae0-9798-95b566adce31 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.460432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 514/1019] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:38.465973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:38.500701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a720d7b-4631-4432-9af7-012f0b6c9b03 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.500895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a720d7b-4631-4432-9af7-012f0b6c9b03 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.501173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a720d7b-4631-4432-9af7-012f0b6c9b03 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.504519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 515/1020] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:38.508575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.508864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.509150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.526949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:38.528170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:38.582268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea983701-1edb-4df2-8f8c-2cc2dbb93aba None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.600723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 516/1021] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:38 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:38.618448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.619349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.619777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.620195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.621118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] RBAC: Adding query filter params (interface=public, service_id=e50996b8f45c4c94b7ee2f34c8dcf9c0, region_id=RegionOne) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:38.621571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.623215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.623735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:38.629933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 506/1022] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/endpoints?service_id=e50996b8f45c4c94b7ee2f34c8dcf9c0&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:38.635217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:38.705497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:38.723463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:38.731139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630a369-f3cf-4c9c-99f8-a77af5eec1e5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.747073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873744bb-7adb-4c02-8093-b9576d1a8aa3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.767319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.768461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.768899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.769182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.769881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.771279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.781426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.782083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.782366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.782643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.783349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:38.783901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:38.784858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 07 13:05:38.786228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:38.787350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.787436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 517/1023] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:38.792394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:38.803294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 507/1024] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:05:38.903609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09e3cd-2461-4d55-9fdf-e09dabf3be2e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:38.940143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:38.940828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:38.941108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:38.941422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:38.942146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:38.942450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:38.944133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:38.954455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 518/1025] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:38.956126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:39.062895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe54ec-c6dc-46fb-94cd-bc4caa493c87 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:39.089696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:39.099038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:39.099703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:39.099992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:39.100268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:39.100978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:39.101281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:39.102644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a2aa5-56a2-41a0-a37a-7e2b11e9ba51 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:39.111088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 508/1026] 10.222.0.61 () {56 vars in 1112 bytes} [Tue Jun 7 13:05:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:39.113743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:39.140514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:39.220202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382d31d-a050-4855-b0c5-ff86055c827d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:39.263738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc2cb-d9eb-44f0-977c-b5f664a7db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:39.264396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc2cb-d9eb-44f0-977c-b5f664a7db8f None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:39.264676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc2cb-d9eb-44f0-977c-b5f664a7db8f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:39.264948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc2cb-d9eb-44f0-977c-b5f664a7db8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:39.265742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc2cb-d9eb-44f0-977c-b5f664a7db8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:39.267091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc2cb-d9eb-44f0-977c-b5f664a7db8f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:39.295218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 519/1027] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:40.489470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba70f13-ac69-42a2-a7f0-308de912f62f None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:40.489991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba70f13-ac69-42a2-a7f0-308de912f62f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:40.490070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba70f13-ac69-42a2-a7f0-308de912f62f None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:40.491670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 509/1028] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:40.497375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:40.497880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:40.497981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:40.514914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:40.516116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:40.590452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 520/1029] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:40.615873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba70f13-ac69-42a2-a7f0-308de912f62f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:40.719915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba70f13-ac69-42a2-a7f0-308de912f62f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:40.755774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9043ec-1c78-487e-bf0c-8b235964d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:40.756591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9043ec-1c78-487e-bf0c-8b235964d87c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:40.756878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9043ec-1c78-487e-bf0c-8b235964d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:40.757158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9043ec-1c78-487e-bf0c-8b235964d87c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:40.758445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9043ec-1c78-487e-bf0c-8b235964d87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:40.759967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9043ec-1c78-487e-bf0c-8b235964d87c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:40.761919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 510/1030] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:40.767612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:40.776755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:40.872145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebfa7f-b340-45d2-92c6-a0646b4d5f43 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:40.906115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50a6fe-619f-4375-8b78-d1c255289556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:40.906790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50a6fe-619f-4375-8b78-d1c255289556 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:40.907103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50a6fe-619f-4375-8b78-d1c255289556 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:40.907377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50a6fe-619f-4375-8b78-d1c255289556 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:40.908097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50a6fe-619f-4375-8b78-d1c255289556 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:40.909547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50a6fe-619f-4375-8b78-d1c255289556 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:40.930904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 521/1031] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:05:40 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:41.085063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87048a2e-5100-474b-9028-49081816d59b None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.085427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87048a2e-5100-474b-9028-49081816d59b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.085613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87048a2e-5100-474b-9028-49081816d59b None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.087241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 511/1032] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:41.092690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.092986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.093261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.111782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:41.113652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:41.175310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:41.189193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 522/1033] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:41.214718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87048a2e-5100-474b-9028-49081816d59b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:41.276966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:41.321407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87048a2e-5100-474b-9028-49081816d59b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:41.358181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:41.358940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.359224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.359495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] PATH_INFO: `/v3/services/image` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.360154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:41.361555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:41.374713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:41.375811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 07 13:05:41.377150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:41.378385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 512/1034] 10.222.0.61 () {56 vars in 1105 bytes} [Tue Jun 7 13:05:41 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:41.382805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:41.483643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f704c-d277-45b2-a448-25f3ec9acb56 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:41.518971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:41.519756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.519934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.520211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.520908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] RBAC: Adding query filter params (name=image) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:41.521211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:41.522565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e05de-116e-468f-842b-021837ef4ece None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:41.532045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 523/1035] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:41.533692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:41.636287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e71ed-029d-4e01-b2e5-5bd21eea1efa None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:41.659108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:41.677125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:41.677929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.678218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.678501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.679263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] RBAC: Adding query filter params (type=image) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:41.679510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e778-e12b-4730-abed-cd8d3037aae4 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.679609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:41.679881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e778-e12b-4730-abed-cd8d3037aae4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.680253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49e778-e12b-4730-abed-cd8d3037aae4 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.680987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:41.681885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 524/1036] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:41.691268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 513/1037] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:41.691405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.691405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.691405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.692295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:41.706896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:41.708300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:41.760287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:41.779815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 525/1038] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:41.801411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b144-24bb-44b9-b390-7bf4ff7719be None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:41.804511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:41.837294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:41.837950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.838233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.838510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.839276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:41.840632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:41.870859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 514/1039] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:05:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:05:41.908712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba8f4b-c4de-4c3e-908a-e2983ba41787 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:41.952177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:41.952999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:41.953284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:41.953562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:41.965517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:41.974561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:41.975629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:41.976496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:41.980846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 526/1040] 10.222.0.61 () {56 vars in 1120 bytes} [Tue Jun 7 13:05:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:41.982075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:42.033026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:42.097210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc022701-43e6-4d3e-8344-69f950a8b151 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:42.134489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:42.135190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:42.135478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:42.135755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:42.136505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:42.136812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:42.138182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:42.147924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 515/1041] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:05:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:42.154983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:42.201685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:42.260810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93817eec-75cf-4876-9322-aec0a2b4946c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:42.295300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:42.296096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:42.296382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:42.296666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:42.307378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:42.308838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:42.325933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:42.327063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:42.328016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:42.328016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:42.328016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:42.328016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:42.330917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 527/1042] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:42.333277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:42.447135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c5d13-9d8d-4b44-8f7b-16e2fada047a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:42.454154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:42.482691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:42.483379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:42.483718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:42.484004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:42.484720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:42.485027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:42.486380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:42.502585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 516/1043] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:42.504356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:42.601138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:42.609802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e9b12-c4c5-491a-aa10-6026993f64eb None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:42.643629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:42.644965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:42.645258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:42.645532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:42.653189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:42.654637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:42.668652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:42.669628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:42.670618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:42.674499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 528/1044] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:05:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:42.675750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:42.778429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef16e3-a77e-4af3-947c-052f4be487c1 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:42.812251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:42.812924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:42.813212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:42.813491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:42.814332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:42.814641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:42.816091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:42.838291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 517/1045] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:05:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:42.839979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:42.946669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08331ce-7625-44ba-a518-9a6efd99fcd4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:42.979569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd673e-08ed-4d83-a9e5-de7c21e8565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:42.980224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd673e-08ed-4d83-a9e5-de7c21e8565a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:42.980510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd673e-08ed-4d83-a9e5-de7c21e8565a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:42.980787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd673e-08ed-4d83-a9e5-de7c21e8565a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.005619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.053166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd673e-08ed-4d83-a9e5-de7c21e8565a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.054846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd673e-08ed-4d83-a9e5-de7c21e8565a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.157293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 529/1046] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:43.163589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.189316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.230525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54b1b-28e2-4d0c-bbe2-aa0ae275607d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.230814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54b1b-28e2-4d0c-bbe2-aa0ae275607d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.231132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54b1b-28e2-4d0c-bbe2-aa0ae275607d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.232634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 530/1047] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:43.237192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.237479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.237760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.254446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:43.255643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:43.269907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95198ea-9114-48ca-a877-e1a3644a7e2c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.304831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.306666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.307751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.308045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.308323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.318288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.320029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.324116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 531/1048] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:43.340453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:43.341362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:43.343874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:43.343874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:43.343874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:43.343874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:43.343874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 518/1049] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:43.347150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.347982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.433608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.459047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5fb2f-a849-45ce-87aa-47143ee11f1c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.459633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafed05-cf13-464e-9ebc-d90f1f8bc110 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.493048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.493687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.493969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.494241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.494992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:43.495309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.496649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.497590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.498765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.499078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.499359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.508013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 532/1050] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:43.513047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:43.513295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:43.521249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:43.522201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:43.523094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 519/1051] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:43.528368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.563665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.622200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06372b9-7afa-4a6b-bbac-2ba4f69151f8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.640362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21728e3-2f53-4c76-b06d-1e6dfda1cf12 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.660954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.661751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.662031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.662301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.669796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.671163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.677869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.678531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.678814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.679163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.679855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:43.680161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.681511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.684177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:43.685472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:43.686646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:43.688750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:43.691832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 533/1052] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:43.692474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.697532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 520/1053] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:43.698576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.798704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992389b-c0b6-427f-bcd5-120ee97a72b2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.805243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873224-f8b1-4021-b7ca-4a0fddd9fd2f None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.808030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.838390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.839235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.839521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.839795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] PATH_INFO: `/v3/users/579a2407609241668bdc6569e93d0050` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.847275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:43.848511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:43.848790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:43.849063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:43.849861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:43.850171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.851627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.855311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] RBAC: Authorizing `identity:get_user(user_id=579a2407609241668bdc6569e93d0050)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:43.856759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:43.863288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 521/1054] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/users/579a2407609241668bdc6569e93d0050 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:43.864994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 534/1055] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:43.865118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.870242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:43.931835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:43.975887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e16b-6e46-43b4-916c-656f3d4f5689 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:43.983495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40072272-633d-4caf-8c6f-f67fe7416438 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:44.010238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.010922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.011207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.011481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.012311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] RBAC: Adding query filter params (role.id=379c934cc3ba4a03bf36ccefb53c4e0e, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=e56bcda01a104f5795f962662cadcd83) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:44.012821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.014043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bd49f-d466-46f4-bd61-05d87fe4468d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.020570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 535/1056] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/role_assignments?user.id=e56bcda01a104f5795f962662cadcd83&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=379c934cc3ba4a03bf36ccefb53c4e0e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:44.022105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.022765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.023085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.023364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.101380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.102791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.157834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb26992-c3cb-492e-bed7-da53e77de0cf None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.158582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb26992-c3cb-492e-bed7-da53e77de0cf None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.158900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb26992-c3cb-492e-bed7-da53e77de0cf None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.160472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 536/1057] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:44.166999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.167335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.167558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.169154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:44.185095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:44.186332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:44.215306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:44.221984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 522/1058] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:44.230012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:44.271593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 537/1059] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:44.293362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:44.354311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ef82f-9221-4ff2-8de2-4501085fa4d2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:44.391830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.392934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.393381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.393822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.395803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] RBAC: Authorizing `identity:get_project(project_id=2abeb162ef264361bd6662837e8bac1c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.397241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.399009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 523/1060] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:05:44 2022] GET /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:44.401947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6be61-437b-4694-933a-13a27e26a037 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:44.404869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:44.437163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:44.444648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.445300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.445588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.445865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.474199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:44.518981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.521705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.523383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfa8dd-a707-4136-9a80-f1d84f702f41 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:44.561248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.562200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.562488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.562762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.563625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=2abeb162ef264361bd6662837e8bac1c, user.id=579a2407609241668bdc6569e93d0050) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:44.563935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.565351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.571580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 524/1061] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:44 2022] GET /identity/v3/role_assignments?user.id=579a2407609241668bdc6569e93d0050&scope.project.id=2abeb162ef264361bd6662837e8bac1c&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:44.630252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 538/1062] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 13:05:44.636363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:44.731427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:44.746710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d7b89-a763-4d27-b05c-471aaad82460 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:44.787607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.788407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.788692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.789124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.799797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.801253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-ae3729c0-82ad-4e47-a38e-1acb31771dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:44.820261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:44.821193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:44.822081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:44.822081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:44.822081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:44.822081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:44.822081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 525/1063] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:44.828428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:44.888536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:44.931582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740162a-f79c-457f-bc38-94bd15aa26a6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:44.966143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:44.966803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:44.967268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:44.967555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:44.968283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:44.968589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:44.969960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6ec1f-4cd1-4f8e-bdab-66bf88b4da78 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:44.981209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 539/1064] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:46.257166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce58dc-4919-4312-940a-96bebf0523ef None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.257706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce58dc-4919-4312-940a-96bebf0523ef None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.257706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce58dc-4919-4312-940a-96bebf0523ef None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.259157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 526/1065] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:46.264476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.264890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.265030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.281957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:46.283162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:46.348182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 540/1066] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:46.370681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce58dc-4919-4312-940a-96bebf0523ef None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:46.472733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce58dc-4919-4312-940a-96bebf0523ef None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:46.509832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:46.510557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.510866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.511148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.518712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:46.522135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:46.530470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:46.531423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:46.532271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 527/1067] 10.222.0.61 () {56 vars in 1120 bytes} [Tue Jun 7 13:05:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:46.537805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:46.585483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:46.634743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b8079-3bec-4906-9a21-aac841a43dbf None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:46.666721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:46.667402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.667682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.667951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.668689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:46.668990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:46.670347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646788bc-7149-4136-b4c0-1f310fd8b882 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:46.678599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 541/1068] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:05:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:46.684405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:46.781611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9ab14-63c3-4f81-a15d-0bde8b1fa80c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:46.799760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349796f9-9aeb-4375-8c19-97007ad1e250 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.800037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349796f9-9aeb-4375-8c19-97007ad1e250 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.800326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349796f9-9aeb-4375-8c19-97007ad1e250 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.802188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 542/1069] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:46.807144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.807436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.807712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.821880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:46.822664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.822983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.823262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.824835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:46.826036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:46.833715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:46.835199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:46.852610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:46.853632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:46.854633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:46.854633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:46.854633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:46.854633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:46.854633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 528/1070] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:46.861588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:46.894346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 543/1071] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:46.914704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:46.917343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:46.958167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:46.962887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ad53d-cadc-4ad8-bb1b-5a335e6b4571 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:46.997085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:46.997738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:46.998028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:46.998304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:46.999054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:46.999361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.000721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.012096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 529/1072] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:47.017773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.019644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7107b0-e587-46d7-80fc-0bffd7b2b9cd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.053770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.054500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.054786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.055090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.064946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:47.072811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:47.073887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:47.075239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 544/1073] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:47.079766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.113064 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a9742-ee27-4216-a267-42e5ac41545a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.126118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:47.146772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.147579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.147861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.148134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.155080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.156540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:47.171507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:47.172576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:47.173558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 530/1074] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:47.174457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:47.177921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef6f75-8d14-458d-b2d7-6c07f45e22ce None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.179192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.213313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.213971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.214249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.214525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.215298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:47.215602 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.216964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.225528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 545/1075] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:47.231536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.279993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c660c5-0a9d-461d-a220-cf73e2f65082 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.313311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.313909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.314198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.314477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.315345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:47.315656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.317080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.329185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce873a-f223-4d69-a92d-9d6136559882 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.334586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 531/1076] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:47.340136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.363963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.364832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.365116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.365393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] PATH_INFO: `/v3/users/579a2407609241668bdc6569e93d0050` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.366593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] RBAC: Authorizing `identity:get_user(user_id=579a2407609241668bdc6569e93d0050)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.368051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.369172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:47.369813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 546/1077] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/users/579a2407609241668bdc6569e93d0050 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:47.375251 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.404459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:47.446725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527291-7b24-43e2-b977-0cfa391facf6 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.476172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b89d69-7d6c-4532-9ffc-532483f62491 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.481856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.482633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.483332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.483610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.493896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.495424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.509311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.510157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.510442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.510714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.511910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] RBAC: Authorizing `identity:get_project(project_id=2abeb162ef264361bd6662837e8bac1c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:47.512552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:47.513559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:47.514591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:47.514591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:47.514591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:47.514591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:47.514816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.514945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 532/1078] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:47.515356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 547/1079] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:47.519881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.522178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.604270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:47.617182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:47.621826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e8447-870d-4c60-ab69-b514e386d716 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.622165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f5360-6c54-4cd2-80eb-55c1503bf74a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.655634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.656176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.656519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.656825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.656918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.657112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.657212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c/users/579a2407609241668bdc6569e93d0050/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.657393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.658115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:47.658423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.659104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2abeb162ef264361bd6662837e8bac1c, user_id=579a2407609241668bdc6569e93d0050, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.659823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.660549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.671807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 533/1080] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:47.674243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 548/1081] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:47 2022] PUT /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c/users/579a2407609241668bdc6569e93d0050/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:05:47.677467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.784116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e037c68-4206-41f2-998d-20c11068189a None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.818985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.819788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.820071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.820341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.828000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.829461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:47.844932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:47.846089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:47.847061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 534/1082] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:47.852147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:47.959179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a53ae-0cdf-415c-a77d-58da2a78fcd3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:47.994690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:47.995392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:47.995676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:47.995949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:47.996732 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:47.997038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:47.998445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb998abc-6b5d-4c36-990a-5b06973c7670 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:48.010787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 549/1083] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:48.016882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:48.019773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:48.115537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9a532-9200-46ff-a8f2-f3b61515ee74 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:48.148617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d7385-c7be-41ca-a9c1-a4bf680fd9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:48.149499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d7385-c7be-41ca-a9c1-a4bf680fd9c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:48.149781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d7385-c7be-41ca-a9c1-a4bf680fd9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:48.150055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d7385-c7be-41ca-a9c1-a4bf680fd9c1 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/e56bcda01a104f5795f962662cadcd83/roles/379c934cc3ba4a03bf36ccefb53c4e0e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:48.152289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d7385-c7be-41ca-a9c1-a4bf680fd9c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=e56bcda01a104f5795f962662cadcd83, role_id=379c934cc3ba4a03bf36ccefb53c4e0e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:48.153723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d7385-c7be-41ca-a9c1-a4bf680fd9c1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:48.166846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 535/1084] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:48 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/e56bcda01a104f5795f962662cadcd83/roles/379c934cc3ba4a03bf36ccefb53c4e0e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:05:49.870242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2a9b8-aca1-450e-8879-458016c26233 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:49.870766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2a9b8-aca1-450e-8879-458016c26233 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:49.870766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2a9b8-aca1-450e-8879-458016c26233 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:49.872168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 550/1085] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:49.877523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:49.877960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:49.878082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:49.894365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:49.895593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:49.951370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:49.963505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 536/1086] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:49.987681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2a9b8-aca1-450e-8879-458016c26233 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.006844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.088936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2a9b8-aca1-450e-8879-458016c26233 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.126016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.126720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.127030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.127307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.137280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:50.144838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:50.145747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:50.146600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 551/1087] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:50.152101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.249903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5be6d-e835-474e-b310-eaa3536feb1a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.284133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.284797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.285116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.285915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.286661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:50.286994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:50.288361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb6634-a871-4fd2-86e1-3a31f749915f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:50.296361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 537/1088] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:50.302134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.378971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.401437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c176ec-6e81-4e9d-87fc-9a20b77847a5 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.424955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a22eb-784f-48d3-91da-11ce00bead1c None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.425247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a22eb-784f-48d3-91da-11ce00bead1c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.425532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a22eb-784f-48d3-91da-11ce00bead1c None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.427059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 538/1089] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:05:50.432153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.432447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.432730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.435696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.436562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.436838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.437108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] PATH_INFO: `/v3/users/579a2407609241668bdc6569e93d0050` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.438278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] RBAC: Authorizing `identity:get_user(user_id=579a2407609241668bdc6569e93d0050)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:50.439738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:50.443649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 552/1090] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/users/579a2407609241668bdc6569e93d0050 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:50.447079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.449494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:50.450684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:50.501247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.520001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 539/1091] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:05:50.543433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.547252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d7022-0fff-4be6-a3d3-947ceb7431be None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.579599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.580245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.580532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.580811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.596204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.641382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81198338-f435-43c4-adc4-14e6f6010249 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.648482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:50.649915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:50.674392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.675244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.675531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.675830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.685894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:05:50.695015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:50.695950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:50.696842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 540/1092] 10.222.0.61 () {56 vars in 1120 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:50.702436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.746380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 553/1093] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 07 13:05:50.747340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.752150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.798395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.802183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ae48c-0502-4c0b-aacb-0c1435fec5fa None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.835999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.836655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.836938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.837217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.837956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:50.838260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:50.839657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:50.847401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 541/1094] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:50.851146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254544c-39cd-46ec-a1f9-5e727adc60da None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.852959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.884991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.885835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.886114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.886383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.887558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] RBAC: Authorizing `identity:get_project(project_id=2abeb162ef264361bd6662837e8bac1c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:50.889001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:50.890674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 554/1095] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:50.897089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:50.952776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886aa65-f00d-48e1-ba3a-1d8080c8ead8 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.960089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:50.986068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:50.986870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:50.987157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:50.987429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:50.997582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a228b6-53ae-49ce-824e-9f84fcd7fe3b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:50.998398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:50.999905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.011669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:51.017340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:51.018277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:51.019221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:51.019221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:51.019221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:51.019221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:51.019221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 542/1096] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:51.024183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:51.032213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:51.032859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:51.033137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:51.033406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:51.034233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=2abeb162ef264361bd6662837e8bac1c, user.id=579a2407609241668bdc6569e93d0050) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:51.034542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:51.035994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.058863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 555/1097] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:50 2022] GET /identity/v3/role_assignments?user.id=579a2407609241668bdc6569e93d0050&scope.project.id=2abeb162ef264361bd6662837e8bac1c&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:51.128686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9e2b5-5669-4dd9-b1c0-ae53a07ea09e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:51.164001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:51.164590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:51.164872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:51.165147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:51.165878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:51.166183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:51.167671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.181755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 543/1098] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:51.187588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:51.289278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ace8a-c986-46d7-92f5-d8fdbee7a19b None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:51.322685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:51.323493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:51.323769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:51.324035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:51.331438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:51.332891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:05:51.346414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:51.348064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:05:51.353890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 556/1099] 10.222.0.61 () {56 vars in 1093 bytes} [Tue Jun 7 13:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:51.356040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:51.375349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:51.454640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7db14-73d0-464f-be5d-4483ab7e29bf None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:51.487542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:51.488799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:51.489095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:51.489368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:51.490201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:51.490505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:51.491961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.508504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 544/1100] 10.222.0.61 () {56 vars in 1133 bytes} [Tue Jun 7 13:05:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:51.514027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:51.553490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:51.612639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff7e03-0d41-4c88-8eaf-7d588bd4586a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:51.646348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:51.647034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:51.647321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:51.647596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:51.714720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:51.716141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.752499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:51.811769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 557/1101] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:51.817449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:51.892405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:51.917191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12e23-a9a0-4127-be64-1736acd17444 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:51.958183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:51.958994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:51.959279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:51.959552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:51.969339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:51.970718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:05:51.988386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:51.989477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:51.990683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:05:51.990683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:05:51.990683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:05:51.990683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:51.994093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 545/1102] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:05:51.995218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:52.098648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4d9cb-fd32-4a26-bf88-fe436bd127d4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:52.131510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:52.132153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:52.132434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:52.132708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:52.133424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:52.133723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:52.135103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:52.141368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:52.147631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 558/1103] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:52.153159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:52.253055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8211b2-b26b-4f44-9431-f60f32734cb3 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:52.280359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:52.288481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:52.289313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:52.289607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:52.289884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:52.296689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:52.298132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:05:52.311935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:05:52.312826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:05:52.313724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 546/1104] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:05:52.319737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:52.416326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870485-a99b-4bb0-8e6d-07bc7547bb92 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:52.452035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:52.452689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:52.452974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:52.453252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:52.454040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:52.454351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:52.455811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d059a-66d6-436b-8290-61c4d9deca40 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:52.468286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 559/1105] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:52.474210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:52.571187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b79f6-43d1-45be-9a77-b40baed2e7be None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:52.603026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:52.603671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:52.603962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:52.604241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:52.605070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] RBAC: Adding query filter params (role.id=379c934cc3ba4a03bf36ccefb53c4e0e, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=e56bcda01a104f5795f962662cadcd83) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:52.605380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:52.606799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38716f61-c336-4475-adb7-494f3679c7e1 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:52.612559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 547/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:52 2022] GET /identity/v3/role_assignments?user.id=e56bcda01a104f5795f962662cadcd83&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=379c934cc3ba4a03bf36ccefb53c4e0e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:53.319853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120191b-af65-4f63-804e-6c2515f86457 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:53.320409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120191b-af65-4f63-804e-6c2515f86457 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:53.320409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120191b-af65-4f63-804e-6c2515f86457 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:53.321751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 560/1107] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:53.327005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:53.327428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:53.327559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:53.344959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:53.346175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:53.395297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:53.417915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 548/1108] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:53.442174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120191b-af65-4f63-804e-6c2515f86457 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:53.502499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:53.543835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120191b-af65-4f63-804e-6c2515f86457 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:53.575830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd6c1b-64bc-4cca-8c3e-4d73da07b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:53.576543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd6c1b-64bc-4cca-8c3e-4d73da07b05a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:53.576817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd6c1b-64bc-4cca-8c3e-4d73da07b05a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:53.577085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd6c1b-64bc-4cca-8c3e-4d73da07b05a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:53.578230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd6c1b-64bc-4cca-8c3e-4d73da07b05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:53.579671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd6c1b-64bc-4cca-8c3e-4d73da07b05a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:53.581257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 561/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:05:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:53.587462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:53.681366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dd9cf-5fbf-4601-84ab-32c6c1781102 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:53.714382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c52341-9044-4368-9d05-5cc6517c8168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:53.715049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c52341-9044-4368-9d05-5cc6517c8168 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:53.715332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c52341-9044-4368-9d05-5cc6517c8168 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:53.715602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c52341-9044-4368-9d05-5cc6517c8168 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:53.716290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c52341-9044-4368-9d05-5cc6517c8168 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:53.717703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c52341-9044-4368-9d05-5cc6517c8168 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:53.736410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 549/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:05:53 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:05:55.975879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be085d5b-e744-4820-93e1-e171ba3e8b94 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:55.976409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be085d5b-e744-4820-93e1-e171ba3e8b94 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:55.976409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be085d5b-e744-4820-93e1-e171ba3e8b94 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:55.977778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 562/1111] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:55.983176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:55.983571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:55.983739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:56.000628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:56.001801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:56.029800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:56.067348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 550/1112] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:56.090984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be085d5b-e744-4820-93e1-e171ba3e8b94 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:56.196321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be085d5b-e744-4820-93e1-e171ba3e8b94 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:56.221286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:56.230368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:56.231176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:56.231457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:56.231730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] PATH_INFO: `/v3/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:56.233181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] RBAC: Authorizing `identity:get_role(role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:56.234539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:56.235902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 563/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:56 2022] GET /identity/v3/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:05:56.241624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:56.342549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2c3c2-be10-48c4-a80f-1dd3dbd2dc28 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:56.375153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:56.376004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:56.376292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:56.376571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] PATH_INFO: `/v3/users/8af725cc4268486c8ee6fa4d6fb96192` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:56.389300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] RBAC: Authorizing `identity:get_user(user_id=8af725cc4268486c8ee6fa4d6fb96192)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:56.390730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:56.392405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 551/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:56 2022] GET /identity/v3/users/8af725cc4268486c8ee6fa4d6fb96192 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:56.398524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:56.493892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620a876-4216-4f34-a90c-3e3b183528b3 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:56.525889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:56.526520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:56.526798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:56.527100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:56.584785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:56.592440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:56.593877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:56.687527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 564/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:05:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:05:56.694224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:56.721367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:56.791554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4973d3-047f-45a4-bd27-5a7869c714b9 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:56.823941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac31fd-6e46-4f54-ad62-582a64832dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:56.824776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac31fd-6e46-4f54-ad62-582a64832dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:56.825059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac31fd-6e46-4f54-ad62-582a64832dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:56.825331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac31fd-6e46-4f54-ad62-582a64832dd6 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:56.826478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac31fd-6e46-4f54-ad62-582a64832dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=2abeb162ef264361bd6662837e8bac1c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:56.827914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac31fd-6e46-4f54-ad62-582a64832dd6 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:56.829575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 552/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:05:56 2022] GET /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:56.836242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:56.924818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:56.931832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4e6f4-cb4d-43c6-843b-094e8b583f8e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:56.965321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:56.965958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:56.966240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:56.966513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:56.967366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] RBAC: Adding query filter params (role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=2abeb162ef264361bd6662837e8bac1c, user.id=8af725cc4268486c8ee6fa4d6fb96192) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:05:56.967675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:56.969087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d283b5-a317-4663-9f5b-e5abc9dc867b None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:56.974491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 565/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:05:56 2022] GET /identity/v3/role_assignments?user.id=8af725cc4268486c8ee6fa4d6fb96192&scope.project.id=2abeb162ef264361bd6662837e8bac1c&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:59.142222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa9a98-4a7c-42e8-91b2-218d29cba01a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:59.142770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa9a98-4a7c-42e8-91b2-218d29cba01a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:59.142770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa9a98-4a7c-42e8-91b2-218d29cba01a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:59.144230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 553/1118] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:05:59.149293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:59.149716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:59.149846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:59.166546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:05:59.167769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:05:59.232624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 566/1119] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:05:59.254629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa9a98-4a7c-42e8-91b2-218d29cba01a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:59.345118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:59.353101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa9a98-4a7c-42e8-91b2-218d29cba01a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:59.384861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:59.385689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:59.385968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:59.386240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] PATH_INFO: `/v3/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:59.387712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] RBAC: Authorizing `identity:get_role(role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:59.389081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:59.390414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 554/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:59 2022] GET /identity/v3/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:59.396722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:59.480083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:05:59.491696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a1c0b-b4a6-4878-afa9-24467fe1ae7a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:59.524334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:59.525200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:59.525486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:59.525764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] PATH_INFO: `/v3/users/8af725cc4268486c8ee6fa4d6fb96192` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:59.526975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] RBAC: Authorizing `identity:get_user(user_id=8af725cc4268486c8ee6fa4d6fb96192)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:59.528413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:59.530039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 567/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:05:59 2022] GET /identity/v3/users/8af725cc4268486c8ee6fa4d6fb96192 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:59.536399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:59.633393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d564b90-539e-48db-9ea9-559b666f017c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:59.665673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb67f0-d746-4708-bbe4-c233ab2e6726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:59.666453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb67f0-d746-4708-bbe4-c233ab2e6726 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:59.666735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb67f0-d746-4708-bbe4-c233ab2e6726 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:59.667038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb67f0-d746-4708-bbe4-c233ab2e6726 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:59.668190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb67f0-d746-4708-bbe4-c233ab2e6726 None admin] RBAC: Authorizing `identity:get_project(project_id=2abeb162ef264361bd6662837e8bac1c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:59.669608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb67f0-d746-4708-bbe4-c233ab2e6726 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:59.671295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 555/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:05:59 2022] GET /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:05:59.676915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:05:59.773623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac7132-f4c7-4f65-9b06-3f324f19a205 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:05:59.806190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16286a-d5b1-4fc0-a3e2-609287d3d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:05:59.807092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16286a-d5b1-4fc0-a3e2-609287d3d099 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:05:59.807374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16286a-d5b1-4fc0-a3e2-609287d3d099 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:05:59.807647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16286a-d5b1-4fc0-a3e2-609287d3d099 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c/users/8af725cc4268486c8ee6fa4d6fb96192/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:05:59.809822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16286a-d5b1-4fc0-a3e2-609287d3d099 None admin] RBAC: Authorizing `identity:create_grant(project_id=2abeb162ef264361bd6662837e8bac1c, user_id=8af725cc4268486c8ee6fa4d6fb96192, role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:05:59.811281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16286a-d5b1-4fc0-a3e2-609287d3d099 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:05:59.824200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 568/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:05:59 2022] PUT /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c/users/8af725cc4268486c8ee6fa4d6fb96192/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:06:02.020685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f37372-e942-4bb2-ba25-11aad12b8b10 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.021223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f37372-e942-4bb2-ba25-11aad12b8b10 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.021223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f37372-e942-4bb2-ba25-11aad12b8b10 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.022609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 556/1124] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:02.027719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.028133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.028277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.044957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:02.046138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:02.057144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:02.111628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 569/1125] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:02.135492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f37372-e942-4bb2-ba25-11aad12b8b10 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:02.171278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:02.241631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f37372-e942-4bb2-ba25-11aad12b8b10 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:02.274143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:02.274953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.275243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.275523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] PATH_INFO: `/v3/roles/2d3b495c407345b2b4637d5b44c3ea39` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.276988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] RBAC: Authorizing `identity:get_role(role_id=2d3b495c407345b2b4637d5b44c3ea39)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:02.278340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:02.279736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 557/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:02 2022] GET /identity/v3/roles/2d3b495c407345b2b4637d5b44c3ea39 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:02.286060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:02.379985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047588dc-ea8a-4b84-a820-81f30dfc9c83 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:02.412898 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:02.413753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.414036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.414311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] PATH_INFO: `/v3/users/8af725cc4268486c8ee6fa4d6fb96192` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.415506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] RBAC: Authorizing `identity:get_user(user_id=8af725cc4268486c8ee6fa4d6fb96192)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:02.416934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:02.418546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 570/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:02 2022] GET /identity/v3/users/8af725cc4268486c8ee6fa4d6fb96192 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:02.424133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:02.515151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:02.518653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4f71a-0da2-4aed-a125-5e0b419f37f5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:02.552603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:02.553179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.553464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.553740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.617353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:02.618770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:02.708467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:02.713420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 558/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:06:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:06:02.719258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:02.734223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:02.813537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91a47c-4568-479b-b58d-e434fb574567 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:02.846368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31071d-ab9e-40b2-9d79-13d3a3c4bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:02.847238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31071d-ab9e-40b2-9d79-13d3a3c4bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.847519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31071d-ab9e-40b2-9d79-13d3a3c4bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.847821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31071d-ab9e-40b2-9d79-13d3a3c4bf69 None admin] PATH_INFO: `/v3/projects/2abeb162ef264361bd6662837e8bac1c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.848977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31071d-ab9e-40b2-9d79-13d3a3c4bf69 None admin] RBAC: Authorizing `identity:get_project(project_id=2abeb162ef264361bd6662837e8bac1c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:02.850393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d31071d-ab9e-40b2-9d79-13d3a3c4bf69 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:02.852071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 571/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:02 2022] GET /identity/v3/projects/2abeb162ef264361bd6662837e8bac1c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:02.857961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:02.952969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739498d0-be31-468c-959c-a2ab36cb2496 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:02.985448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:02.986085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:02.986371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:02.986647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:02.987498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] RBAC: Adding query filter params (role.id=2d3b495c407345b2b4637d5b44c3ea39, scope.project.id=2abeb162ef264361bd6662837e8bac1c, user.id=8af725cc4268486c8ee6fa4d6fb96192) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:02.987809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:02.989721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d433-5f21-46c8-8de7-12b9114326c8 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:02.995514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 559/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:06:02 2022] GET /identity/v3/role_assignments?user.id=8af725cc4268486c8ee6fa4d6fb96192&scope.project.id=2abeb162ef264361bd6662837e8bac1c&role.id=2d3b495c407345b2b4637d5b44c3ea39 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:05.221491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ebfd4-ba91-43fc-bd9a-4e3eb9fa2ec5 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:05.222057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ebfd4-ba91-43fc-bd9a-4e3eb9fa2ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:05.222057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ebfd4-ba91-43fc-bd9a-4e3eb9fa2ec5 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:05.224049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 572/1131] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:05.229250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:05.229689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:05.229800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:05.246091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:05.247307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:05.302495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:05.313067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 560/1132] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:05.336747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ebfd4-ba91-43fc-bd9a-4e3eb9fa2ec5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:05.416672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:05.436254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ebfd4-ba91-43fc-bd9a-4e3eb9fa2ec5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:05.469751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fe29b-3943-4350-9c42-5557d75d0793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:05.470522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fe29b-3943-4350-9c42-5557d75d0793 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:05.470799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fe29b-3943-4350-9c42-5557d75d0793 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:05.471101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fe29b-3943-4350-9c42-5557d75d0793 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:05.472293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fe29b-3943-4350-9c42-5557d75d0793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:05.474367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fe29b-3943-4350-9c42-5557d75d0793 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:05.476064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 573/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:05.481964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:05.577007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c489b1d-834b-4458-a3c2-cd0791a20325 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:05.610137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b4002-a80f-433c-83bf-1feb816274c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:05.610785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4002-a80f-433c-83bf-1feb816274c3 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:05.611103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4002-a80f-433c-83bf-1feb816274c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:05.611383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4002-a80f-433c-83bf-1feb816274c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:05.612090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b4002-a80f-433c-83bf-1feb816274c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:05.613543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b4002-a80f-433c-83bf-1feb816274c3 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:05.637699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 561/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:06:05 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:06:07.892453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651acbfb-e754-463b-ad16-342c12d18aa0 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:07.893047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651acbfb-e754-463b-ad16-342c12d18aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:07.893047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651acbfb-e754-463b-ad16-342c12d18aa0 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:07.894350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 574/1135] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:07.899607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:07.900010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:07.900177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:07.917087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:07.918266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:07.957189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:07.985057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 562/1136] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:08.010333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651acbfb-e754-463b-ad16-342c12d18aa0 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:08.107754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651acbfb-e754-463b-ad16-342c12d18aa0 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:08.139662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df563eb-f503-4495-852f-ea8b62a62fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:08.140427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df563eb-f503-4495-852f-ea8b62a62fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:08.140705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df563eb-f503-4495-852f-ea8b62a62fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:08.140971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df563eb-f503-4495-852f-ea8b62a62fc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:08.142140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df563eb-f503-4495-852f-ea8b62a62fc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:08.143618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df563eb-f503-4495-852f-ea8b62a62fc9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:08.145726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 575/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:08.151162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:08.247008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b1485-f282-4409-9449-cad4c815be61 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:08.279825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bfa5d-62c1-49b9-855f-2736297dd753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:08.280485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bfa5d-62c1-49b9-855f-2736297dd753 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:08.280774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bfa5d-62c1-49b9-855f-2736297dd753 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:08.281054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bfa5d-62c1-49b9-855f-2736297dd753 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:08.281754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bfa5d-62c1-49b9-855f-2736297dd753 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:08.283214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bfa5d-62c1-49b9-855f-2736297dd753 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:08.303221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 563/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:06:08 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:06:10.583740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8918c-d2ad-4099-87e4-e58833a0092c None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:10.584253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8918c-d2ad-4099-87e4-e58833a0092c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:10.584371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8918c-d2ad-4099-87e4-e58833a0092c None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:10.585634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 576/1139] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:10.590779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:10.591218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:10.591364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:10.607707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:10.608886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:10.632607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:10.674251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 564/1140] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:06:10.699127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8918c-d2ad-4099-87e4-e58833a0092c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:10.717372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:10.796948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8918c-d2ad-4099-87e4-e58833a0092c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:10.829811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:10.830587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:10.830906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:10.831183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:10.840828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:06:10.848887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:10.849976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:06:10.851055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 577/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:10.856059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:10.950560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74423a7-d12f-4337-bf9d-54c14fd9990b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:10.983090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:10.983741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:10.984023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:10.984299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:10.985045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:10.985350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:10.986706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:10.995151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 565/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:11.000538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:11.075678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:11.096515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7681aad-a123-4534-b6c2-5544e8ca1522 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:11.128935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:11.129785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:11.130062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:11.130333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] PATH_INFO: `/v3/users/d3986867410a4a2fbdc63855619ac6b8` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:11.142907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] RBAC: Authorizing `identity:get_user(user_id=d3986867410a4a2fbdc63855619ac6b8)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:11.144351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:11.145990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 578/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:10 2022] GET /identity/v3/users/d3986867410a4a2fbdc63855619ac6b8 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:11.151487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:11.176445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:11.250746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b27f55-4bd7-4f03-ae8f-39023c53821c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:11.282695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:11.283353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:11.283637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:11.283910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:11.348968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:11.350407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:11.372107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:11.445044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 566/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:06:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:06:11.450727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:11.546133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d69930-6a35-46d3-88d3-3f38e3d3abb8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:11.578454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4f3f-ef43-4c93-a829-9eb511126d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:11.579332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4f3f-ef43-4c93-a829-9eb511126d4f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:11.579616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4f3f-ef43-4c93-a829-9eb511126d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:11.579887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4f3f-ef43-4c93-a829-9eb511126d4f None admin] PATH_INFO: `/v3/projects/fed2ba47241341ecb437cb65e66ab770` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:11.581055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4f3f-ef43-4c93-a829-9eb511126d4f None admin] RBAC: Authorizing `identity:get_project(project_id=fed2ba47241341ecb437cb65e66ab770)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:11.583005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4f3f-ef43-4c93-a829-9eb511126d4f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:11.584679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 579/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:11 2022] GET /identity/v3/projects/fed2ba47241341ecb437cb65e66ab770 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:11.590485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:11.684647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74339c-cd4a-4a37-baa7-1c347cf02c78 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:11.706584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:11.718807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:11.719501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:11.719784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:11.720057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:11.720885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=fed2ba47241341ecb437cb65e66ab770, user.id=d3986867410a4a2fbdc63855619ac6b8) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:11.721191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:11.722604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b9810-d370-4ac8-a7c8-374f1724d197 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:11.727905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 567/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:06:11 2022] GET /identity/v3/role_assignments?user.id=d3986867410a4a2fbdc63855619ac6b8&scope.project.id=fed2ba47241341ecb437cb65e66ab770&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:13.901260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bb7ba-a8cb-47de-8db0-e5abc2c5d47e None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:13.901812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bb7ba-a8cb-47de-8db0-e5abc2c5d47e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:13.901812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bb7ba-a8cb-47de-8db0-e5abc2c5d47e None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:13.903212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 580/1147] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:13.908249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:13.908659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:13.908801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:13.925066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:13.926245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:13.989977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 568/1148] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:14.021088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bb7ba-a8cb-47de-8db0-e5abc2c5d47e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:14.024886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:14.117765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bb7ba-a8cb-47de-8db0-e5abc2c5d47e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:14.150314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:14.151112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:14.151439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:14.151663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:14.161298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:06:14.168729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:14.169850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:06:14.170741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 581/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:14.175667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:14.270817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eee0e5-5f12-46f6-b953-9ffe0444544b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:14.286398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:14.304155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:14.304818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:14.305107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:14.305387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:14.306140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:14.306446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:14.307841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:14.315647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 569/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:06:14.322276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:14.382661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:14.420056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c4bd6-622b-4bd0-b278-2522bc1666bd None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:14.453676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:14.454541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:14.454843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:14.455132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] PATH_INFO: `/v3/users/d3986867410a4a2fbdc63855619ac6b8` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:14.456312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] RBAC: Authorizing `identity:get_user(user_id=d3986867410a4a2fbdc63855619ac6b8)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:14.457745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:14.459395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 582/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:14 2022] GET /identity/v3/users/d3986867410a4a2fbdc63855619ac6b8 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:14.464989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:14.560995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b43d02-3cc6-475f-a7bd-b8905ad21a18 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:14.593783 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64f75f-786f-4f19-b5bd-4f24b9f4f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:14.594620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64f75f-786f-4f19-b5bd-4f24b9f4f279 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:14.594931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64f75f-786f-4f19-b5bd-4f24b9f4f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:14.595211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64f75f-786f-4f19-b5bd-4f24b9f4f279 None admin] PATH_INFO: `/v3/projects/fed2ba47241341ecb437cb65e66ab770` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:14.596367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64f75f-786f-4f19-b5bd-4f24b9f4f279 None admin] RBAC: Authorizing `identity:get_project(project_id=fed2ba47241341ecb437cb65e66ab770)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:14.598319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64f75f-786f-4f19-b5bd-4f24b9f4f279 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:14.599993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 570/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:14 2022] GET /identity/v3/projects/fed2ba47241341ecb437cb65e66ab770 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:14.605488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:14.701960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1771f-4b0f-4f03-84c0-c205051b0467 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:14.729492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:14.734744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae192a62-a176-42b4-b863-b9c1c61044dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:14.735648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192a62-a176-42b4-b863-b9c1c61044dd None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:14.735929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192a62-a176-42b4-b863-b9c1c61044dd None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:14.736203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192a62-a176-42b4-b863-b9c1c61044dd None admin] PATH_INFO: `/v3/projects/fed2ba47241341ecb437cb65e66ab770/users/d3986867410a4a2fbdc63855619ac6b8/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:14.738140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae192a62-a176-42b4-b863-b9c1c61044dd None admin] RBAC: Authorizing `identity:create_grant(project_id=fed2ba47241341ecb437cb65e66ab770, user_id=d3986867410a4a2fbdc63855619ac6b8, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:14.740177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae192a62-a176-42b4-b863-b9c1c61044dd None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:14.752321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 583/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:06:14 2022] PUT /identity/v3/projects/fed2ba47241341ecb437cb65e66ab770/users/d3986867410a4a2fbdc63855619ac6b8/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:06:16.939441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd770c9-a77b-4b87-b5e9-13e0094526df None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:16.939963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd770c9-a77b-4b87-b5e9-13e0094526df None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:16.940055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd770c9-a77b-4b87-b5e9-13e0094526df None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:16.941361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 571/1154] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:16.946799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:16.947222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:16.947380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:16.963647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:16.964839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:17.028152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 584/1155] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:17.052246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd770c9-a77b-4b87-b5e9-13e0094526df None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:17.099150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:17.151019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd770c9-a77b-4b87-b5e9-13e0094526df None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:17.183557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:17.184327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:17.184612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:17.184889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:17.195428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:06:17.202864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:17.203899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:06:17.205112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 572/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:17.210344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:17.305650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d8e69-bf13-4a1a-b33e-196bf0f80ea2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:17.327372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:17.340270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:17.340921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:17.341206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:17.341482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:17.342225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:17.342528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:17.343915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:17.352263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 585/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:17.357773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:17.453896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb24dd-c4dd-46be-8991-554fac3aa659 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:17.461341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:17.487622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:17.488477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:17.488792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:17.489071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] PATH_INFO: `/v3/users/d3986867410a4a2fbdc63855619ac6b8` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:17.490241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d3986867410a4a2fbdc63855619ac6b8)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:17.491709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:17.493327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 573/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:17 2022] GET /identity/v3/users/d3986867410a4a2fbdc63855619ac6b8 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:17.498965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:17.601640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9b3a6-43a3-42f5-ada5-acc77669038e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:17.636080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:17.636728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:17.637016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:17.637293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:17.686982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:17.703180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:17.704668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:17.799674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 586/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:06:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:06:17.805492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:17.900861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec5ca3-fa3f-4c7c-8c4b-e481811907f5 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:17.932955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e46a-add3-4c13-b43c-77770abf0e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:17.933790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e46a-add3-4c13-b43c-77770abf0e96 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:17.934073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e46a-add3-4c13-b43c-77770abf0e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:17.934346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e46a-add3-4c13-b43c-77770abf0e96 None admin] PATH_INFO: `/v3/projects/fed2ba47241341ecb437cb65e66ab770` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:17.935550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127e46a-add3-4c13-b43c-77770abf0e96 None admin] RBAC: Authorizing `identity:get_project(project_id=fed2ba47241341ecb437cb65e66ab770)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:17.936971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127e46a-add3-4c13-b43c-77770abf0e96 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:17.938642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 574/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:17 2022] GET /identity/v3/projects/fed2ba47241341ecb437cb65e66ab770 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:17.945329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:18.026636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:18.042606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a4e8-c1a1-4f86-800a-61d659b63c16 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:18.076679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:18.077327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:18.077612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:18.077890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:18.078719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=fed2ba47241341ecb437cb65e66ab770, user.id=d3986867410a4a2fbdc63855619ac6b8) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:18.079061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:18.080496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8c7b-d011-4e3a-a118-f7d3c85b5797 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:18.086304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 587/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:06:17 2022] GET /identity/v3/role_assignments?user.id=d3986867410a4a2fbdc63855619ac6b8&scope.project.id=fed2ba47241341ecb437cb65e66ab770&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:20.325659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b244c-b63e-4531-9535-8bcadf99d520 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:20.326213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b244c-b63e-4531-9535-8bcadf99d520 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:20.326213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b244c-b63e-4531-9535-8bcadf99d520 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:20.327621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 575/1162] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:20.333377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:20.333794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:20.333934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:20.351213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:20.352418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:20.416962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 588/1163] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:20.438914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b244c-b63e-4531-9535-8bcadf99d520 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:20.483706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:20.538683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b244c-b63e-4531-9535-8bcadf99d520 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:20.571211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:20.571841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:20.572127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:20.572399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:20.636549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:20.637929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:20.677127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:20.732585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 576/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:06:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:06:20.738413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:20.813004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:20.835243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc1742-1551-4bc9-ab46-c126c0e546e0 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:20.867418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:20.868204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:20.868487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:20.868763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:20.875279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:20.876720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-324046f6-804e-49c1-b22d-913923b6089a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:20.891194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:20.892410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:06:20.893506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 589/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:06:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:20.898684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:20.995082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15fa13-901d-4fcf-b491-2cc58e080e93 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:21.027787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:21.028427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:21.028718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:21.028995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:21.029710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:21.030017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:21.031408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade8c29-147e-41f5-bdcb-002e09375331 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:21.038379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 577/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Tue Jun 7 13:06:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:23.247502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaff79-21b5-4323-91d1-ddd78c70a9f5 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:23.248034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaff79-21b5-4323-91d1-ddd78c70a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:23.248034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaff79-21b5-4323-91d1-ddd78c70a9f5 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:23.249413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 590/1167] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:23.255159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3349-b0b0-4403-8aa4-451f6d4d755e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:23.255554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3349-b0b0-4403-8aa4-451f6d4d755e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:23.255719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3349-b0b0-4403-8aa4-451f6d4d755e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:23.266994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:23.272446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-34cd3349-b0b0-4403-8aa4-451f6d4d755e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:23.273623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-34cd3349-b0b0-4403-8aa4-451f6d4d755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:23.337490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 578/1168] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:23.359900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaff79-21b5-4323-91d1-ddd78c70a9f5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:23.462473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaff79-21b5-4323-91d1-ddd78c70a9f5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:23.494182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754cf12-a8cc-4387-bfb5-8b7f3b6c69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:23.494821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754cf12-a8cc-4387-bfb5-8b7f3b6c69ef None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:23.495132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754cf12-a8cc-4387-bfb5-8b7f3b6c69ef None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:23.495407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754cf12-a8cc-4387-bfb5-8b7f3b6c69ef None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:23.496144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754cf12-a8cc-4387-bfb5-8b7f3b6c69ef None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:23.497488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754cf12-a8cc-4387-bfb5-8b7f3b6c69ef None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:23.506293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:23.513232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 591/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Tue Jun 7 13:06:23 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:06:25.729204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203918d-8c51-4e7b-9d5a-d1f98119e07d None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:25.729724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203918d-8c51-4e7b-9d5a-d1f98119e07d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:25.729724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203918d-8c51-4e7b-9d5a-d1f98119e07d None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:25.731076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 579/1170] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:25.736307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:25.736705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:25.736867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:25.753687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:25.755499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:25.819777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 592/1171] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:25.843396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203918d-8c51-4e7b-9d5a-d1f98119e07d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:25.941914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203918d-8c51-4e7b-9d5a-d1f98119e07d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:25.974410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dd175-de09-4df9-a7c5-b877151ee911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:25.975281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dd175-de09-4df9-a7c5-b877151ee911 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:25.975565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dd175-de09-4df9-a7c5-b877151ee911 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:25.975839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dd175-de09-4df9-a7c5-b877151ee911 None admin] PATH_INFO: `/v3/domains/202990de687546f19d1bf42d66a8ada9` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:25.978469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:25.986840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dd175-de09-4df9-a7c5-b877151ee911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=202990de687546f19d1bf42d66a8ada9)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:25.988289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dd175-de09-4df9-a7c5-b877151ee911 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:25.990507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 580/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Tue Jun 7 13:06:25 2022] GET /identity/v3/domains/202990de687546f19d1bf42d66a8ada9 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:25.996466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:26.091863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9b9b6-d695-4492-bda5-f862d42f006c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:26.118890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:26.126801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057c855-2c43-4337-b925-b975c11ed936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:26.127478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057c855-2c43-4337-b925-b975c11ed936 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:26.127762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057c855-2c43-4337-b925-b975c11ed936 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:26.128040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057c855-2c43-4337-b925-b975c11ed936 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:26.128739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057c855-2c43-4337-b925-b975c11ed936 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:26.130186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057c855-2c43-4337-b925-b975c11ed936 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:26.153937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 593/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:06:25 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:06:28.403328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148fe63-7ab3-4950-9b51-cbf71ddb0205 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:28.403906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148fe63-7ab3-4950-9b51-cbf71ddb0205 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:28.403906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148fe63-7ab3-4950-9b51-cbf71ddb0205 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:28.405266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 581/1174] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:28.410424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:28.410859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:28.411061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:28.428381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:28.429573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:28.494912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 594/1175] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:28.520122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148fe63-7ab3-4950-9b51-cbf71ddb0205 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:28.617486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148fe63-7ab3-4950-9b51-cbf71ddb0205 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:28.649136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf3e69-5367-4deb-add0-af2296940ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:28.649974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf3e69-5367-4deb-add0-af2296940ede None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:28.650253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf3e69-5367-4deb-add0-af2296940ede None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:28.650524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf3e69-5367-4deb-add0-af2296940ede None admin] PATH_INFO: `/v3/domains/202990de687546f19d1bf42d66a8ada9` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:28.651695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf3e69-5367-4deb-add0-af2296940ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=202990de687546f19d1bf42d66a8ada9)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:28.653123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf3e69-5367-4deb-add0-af2296940ede None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:28.654736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 582/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Tue Jun 7 13:06:28 2022] GET /identity/v3/domains/202990de687546f19d1bf42d66a8ada9 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:28.660301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:28.756675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b37e8-f6e5-4c1c-b426-2052351609fa None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:28.766812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:28.790003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb89b19-2d03-463f-ba9c-682ffbc88a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:28.790660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb89b19-2d03-463f-ba9c-682ffbc88a3e None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:28.790973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb89b19-2d03-463f-ba9c-682ffbc88a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:28.791253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb89b19-2d03-463f-ba9c-682ffbc88a3e None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:28.791962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb89b19-2d03-463f-ba9c-682ffbc88a3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:28.793384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb89b19-2d03-463f-ba9c-682ffbc88a3e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:28.813281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 595/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:06:28 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:06:31.068825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35052015-efe2-424a-b847-2045f07b4391 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:31.069359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35052015-efe2-424a-b847-2045f07b4391 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:31.069359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35052015-efe2-424a-b847-2045f07b4391 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:31.071024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 583/1178] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:31.076264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:31.076686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:31.076824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:31.093871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:31.095081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:31.159004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 596/1179] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:31.181519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35052015-efe2-424a-b847-2045f07b4391 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:31.244309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:31.281688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35052015-efe2-424a-b847-2045f07b4391 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:31.312825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:31.313599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:31.313880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:31.314154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:31.325106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:06:31.333093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:31.334216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:06:31.335516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 584/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:31.340248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:31.429338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:31.436425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093d3f2-bc29-4e01-a2c9-3c11b8e992d1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:31.470357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:31.471039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:31.471324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:31.471600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:31.472348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:31.472653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:31.474012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:31.481792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 597/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:31.487464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:31.582740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e78700-b2cd-4f7f-ada7-6648b1909cf2 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:31.605426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:31.616584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:31.617428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:31.617705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:31.617977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] PATH_INFO: `/v3/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:31.631002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:31.632436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:31.634038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 585/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:31 2022] GET /identity/v3/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:31.639610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:31.735459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b08e04-441d-4041-b1b0-87208fa815f9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:31.769037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:31.769664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:31.769981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:31.770263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:31.795818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:31.836297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:31.837685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:31.933346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 598/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:06:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:06:31.939057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:32.033790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb139e-d6c6-4b6d-b878-de66e9c593c0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:32.066578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee0d3d-957b-491f-8ebc-98b39ce82230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:32.067457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee0d3d-957b-491f-8ebc-98b39ce82230 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:32.067738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee0d3d-957b-491f-8ebc-98b39ce82230 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:32.068010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee0d3d-957b-491f-8ebc-98b39ce82230 None admin] PATH_INFO: `/v3/projects/6e856b80151a452cbb97716fe61e62d6` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:32.069169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee0d3d-957b-491f-8ebc-98b39ce82230 None admin] RBAC: Authorizing `identity:get_project(project_id=6e856b80151a452cbb97716fe61e62d6)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:32.070852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee0d3d-957b-491f-8ebc-98b39ce82230 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:32.072530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 586/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:31 2022] GET /identity/v3/projects/6e856b80151a452cbb97716fe61e62d6 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:32.078414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:32.134017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:32.173510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019877e8-ce43-462c-b62f-2f90b8519d70 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:32.206102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:32.206740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:32.207048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:32.207328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:32.208682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=6e856b80151a452cbb97716fe61e62d6, user.id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:32.208992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:32.210414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b654f-5216-4d6b-a07b-1687d12c49b8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:32.215739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 599/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:06:32 2022] GET /identity/v3/role_assignments?user.id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6&scope.project.id=6e856b80151a452cbb97716fe61e62d6&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:34.434726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe0cc7-13e2-4cfe-a45d-9ebdde78cf07 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:34.435172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe0cc7-13e2-4cfe-a45d-9ebdde78cf07 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:34.435307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe0cc7-13e2-4cfe-a45d-9ebdde78cf07 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:34.436682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 587/1186] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:34.441960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:34.442365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:34.442513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:34.459538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:34.460717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:34.524659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 600/1187] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:34.547467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe0cc7-13e2-4cfe-a45d-9ebdde78cf07 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:34.597061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:34.646764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe0cc7-13e2-4cfe-a45d-9ebdde78cf07 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:34.679462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:34.680236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:34.680575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:34.680798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:34.690581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:06:34.698539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:34.699699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:06:34.700890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 588/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:34.705585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:34.755828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:34.801256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe4a4b-dd76-4361-afba-a2b51c050c20 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:34.834332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:34.835004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:34.835288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:34.835564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:34.836308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:34.836612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:34.837966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:34.845785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 601/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:34.851621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:34.948771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eeea1-194e-4cb0-8ae5-30ca59e9f6d0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:34.959096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:34.981746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:34.982597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:34.982921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:34.983202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] PATH_INFO: `/v3/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:34.984375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] RBAC: Authorizing `identity:get_user(user_id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:34.985800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:34.987433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 589/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:34 2022] GET /identity/v3/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:34.992887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:35.089424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ce849-8e84-46ae-82bb-9009ac8b6466 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:35.102250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:35.121591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de8686a-0e57-4abd-9e35-3b794bece205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:35.122427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8686a-0e57-4abd-9e35-3b794bece205 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:35.122710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8686a-0e57-4abd-9e35-3b794bece205 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:35.123014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de8686a-0e57-4abd-9e35-3b794bece205 None admin] PATH_INFO: `/v3/projects/6e856b80151a452cbb97716fe61e62d6` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:35.124176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8686a-0e57-4abd-9e35-3b794bece205 None admin] RBAC: Authorizing `identity:get_project(project_id=6e856b80151a452cbb97716fe61e62d6)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:35.125589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de8686a-0e57-4abd-9e35-3b794bece205 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:35.127266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 602/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:34 2022] GET /identity/v3/projects/6e856b80151a452cbb97716fe61e62d6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:35.133841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:35.230948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6312-5ff2-4327-8a53-3008f1542ccf None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:35.263626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1a1b5-3baa-4500-8479-c33eac552ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:35.264494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1a1b5-3baa-4500-8479-c33eac552ffb None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:35.264779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1a1b5-3baa-4500-8479-c33eac552ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:35.265054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1a1b5-3baa-4500-8479-c33eac552ffb None admin] PATH_INFO: `/v3/projects/6e856b80151a452cbb97716fe61e62d6/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:35.267018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1a1b5-3baa-4500-8479-c33eac552ffb None admin] RBAC: Authorizing `identity:create_grant(project_id=6e856b80151a452cbb97716fe61e62d6, user_id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:35.268436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1a1b5-3baa-4500-8479-c33eac552ffb None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:35.280691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 590/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:06:35 2022] PUT /identity/v3/projects/6e856b80151a452cbb97716fe61e62d6/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:06:37.473868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c64e0-5b60-4ce6-95c6-c23203e83c03 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:37.474315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c64e0-5b60-4ce6-95c6-c23203e83c03 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:37.474416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c64e0-5b60-4ce6-95c6-c23203e83c03 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:37.475802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 603/1193] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:06:37.480899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:37.481269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:37.481457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:37.497794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:06:37.499004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:06:37.527620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:37.564798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 591/1194] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:06:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:06:37.593402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c64e0-5b60-4ce6-95c6-c23203e83c03 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:37.691429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c64e0-5b60-4ce6-95c6-c23203e83c03 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:37.724269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:37.725037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:37.725315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:37.725584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:37.736400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:06:37.743928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:06:37.745002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:06:37.745903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 604/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:06:37.750999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:37.845423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebb1df-65f6-4473-b633-94a0b400a000 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:37.876914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:37.877562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:37.877841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:37.878113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:37.878868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:37.879173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:37.881168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:37.889056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 592/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:37.894890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:37.936992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:37.991114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0ace5-b280-4114-b6aa-c6685cccbd4d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:38.029268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:38.030127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:38.030406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:38.030677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] PATH_INFO: `/v3/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:38.031868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] RBAC: Authorizing `identity:get_user(user_id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:38.033296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:38.034921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 605/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Tue Jun 7 13:06:37 2022] GET /identity/v3/users/d5f97bcb4b7e4ff3b0bcbd3235c4d9f6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:38.040702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:38.070700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:38.144444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373a31c-20fc-4b56-953a-7bdcbd55b9c6 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:38.177414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:38.178043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:38.178324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:38.178595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:38.245479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:38.246876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:38.274720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:38.342164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 593/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:06:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:06:38.348207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:38.443760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5ad32-3b7d-4901-ae85-3b96c7a8c2fc None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:38.454257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:38.477124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c16428-1ecb-43a2-a6ec-00a51680e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:38.477969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16428-1ecb-43a2-a6ec-00a51680e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:38.478250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16428-1ecb-43a2-a6ec-00a51680e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:38.478526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c16428-1ecb-43a2-a6ec-00a51680e3d2 None admin] PATH_INFO: `/v3/projects/6e856b80151a452cbb97716fe61e62d6` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:38.479717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c16428-1ecb-43a2-a6ec-00a51680e3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=6e856b80151a452cbb97716fe61e62d6)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:38.481137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c16428-1ecb-43a2-a6ec-00a51680e3d2 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:38.482805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 606/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Tue Jun 7 13:06:38 2022] GET /identity/v3/projects/6e856b80151a452cbb97716fe61e62d6 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:38.489472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:06:38.585006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420eea-d952-4558-b933-9a81794fe6d7 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:06:38.612266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:38.618776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:06:38.619438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:06:38.619724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:06:38.620000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:06:38.620827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=6e856b80151a452cbb97716fe61e62d6, user.id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:06:38.621137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:06:38.622556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbad0ec-ee81-404e-acd1-7f005f1fce93 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:06:38.628290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 594/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:06:38 2022] GET /identity/v3/role_assignments?user.id=d5f97bcb4b7e4ff3b0bcbd3235c4d9f6&scope.project.id=6e856b80151a452cbb97716fe61e62d6&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:06:48.473184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=11 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:06:48.625820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:38.937536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072e60-a4bb-4b28-bed9-d7be1fffcf47 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:38.938261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072e60-a4bb-4b28-bed9-d7be1fffcf47 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:38.938261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35072e60-a4bb-4b28-bed9-d7be1fffcf47 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:38.939464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 607/1201] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:07:38.948001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:38.948483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:38.950205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:38.971286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:07:38.972612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:07:39.059620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 595/1202] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:07:39.087580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072e60-a4bb-4b28-bed9-d7be1fffcf47 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:39.234846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35072e60-a4bb-4b28-bed9-d7be1fffcf47 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:39.294065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:39.295401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:39.295857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:39.296350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:39.312219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:39.313766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:39.339198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:39.340689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:39.341676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:39.341676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:39.341676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:39.341676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:39.341676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 608/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:39.346310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:39.471040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf5194-62ab-497d-8a2b-ba2fe025b058 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:39.509143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:39.516915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:39.517610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:39.517907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:39.518181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:39.518973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:39.519350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:39.520687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148cead9-e2fd-45a1-99af-6e6463360e59 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:39.538332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 596/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:39.540924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:39.609751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:39.671386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787e767-1d2e-4548-b4d6-878370cf3f98 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:39.714393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a68eb-7a8a-40ae-9a62-bfda930cfb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:39.715469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a68eb-7a8a-40ae-9a62-bfda930cfb90 None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:39.716044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a68eb-7a8a-40ae-9a62-bfda930cfb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:39.716530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a68eb-7a8a-40ae-9a62-bfda930cfb90 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:39.717434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a68eb-7a8a-40ae-9a62-bfda930cfb90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:39.720173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a68eb-7a8a-40ae-9a62-bfda930cfb90 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:39.746882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 609/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Tue Jun 7 13:07:39 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:07:42.332644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d332-7073-4ae9-912c-447f00ecb876 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:42.333240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d332-7073-4ae9-912c-447f00ecb876 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:42.333240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d332-7073-4ae9-912c-447f00ecb876 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:42.335509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 597/1206] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:07:42.341579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:42.342108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:42.342223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:42.360276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:07:42.361471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:07:42.436890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 610/1207] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:07:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:07:42.461774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d332-7073-4ae9-912c-447f00ecb876 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:42.590955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d332-7073-4ae9-912c-447f00ecb876 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:42.632878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:42.634346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:42.634856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:42.635358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:42.648986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:07:42.656946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:42.658121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:42.659417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 598/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:07:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:42.664815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:42.721609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:42.792920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89436a30-b995-4aa9-a223-e3880b96e5fe None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:42.835803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:42.836477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:42.836858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:42.837167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:42.838007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:42.838317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:42.839880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:42.849568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 611/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:07:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:42.855548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:42.912031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:42.977322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3ccdf-966c-4efe-8f7d-707f8a44e462 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:43.017346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:43.018180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:43.018469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:43.018747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:43.030968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:43.032662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:43.055670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:43.056812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:43.057722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:43.057722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:43.057722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:43.057722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:43.057722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 599/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:07:43.065876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:43.181579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:43.184671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d680-9d54-4817-a63a-6f0fe3e116c4 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:43.221631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:43.222680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:43.223016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:43.223351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:43.224524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:43.224833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:43.226195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:43.240208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 612/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:43.246133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:43.358982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bf8ac-3539-4bc9-a166-e274c688eb17 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:43.360669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:43.398243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:43.399078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:43.399378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:43.399637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:43.408881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:43.410343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:07:43.425656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:43.426752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:43.428911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 600/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:43.433520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:43.557164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2940b0-6fed-41ea-a702-4c79a129182d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:43.601235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:43.601953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:43.602261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:43.602553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:43.603496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:43.603804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:43.605376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:43.625535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 613/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Tue Jun 7 13:07:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:43.631541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:43.743691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546b253-a98a-49dc-87f4-7f3895743e78 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:43.781672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:43.782483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:43.782896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:43.783276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:43.828933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:43.877720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:43.879232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:44.001024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 601/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:07:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:07:44.002848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:44.025985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:44.116532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8288841-7ce2-4afc-b2ce-4e51cb794bad None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:44.155417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:44.156353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:44.156761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:44.157255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:44.172697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:44.175100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:44.202009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:44.203296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:44.204407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:44.204407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:44.204407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:44.204407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:44.209621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 614/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:44.211638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:44.299237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:44.351127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80691770-6252-4d4f-ab9b-2394cd313758 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:44.387924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:44.388592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:44.388881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:44.389158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:44.389888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:44.390214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:44.391598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:44.403678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 602/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:44.409151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:44.491131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:44.527723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57b384-9e7e-4226-b83c-e9516e81d5ea None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:44.569564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:44.569564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:44.569564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:44.569948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:44.577303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:44.578770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:07:44.592671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:44.593675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:44.595765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 615/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:44.601286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:44.723255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f51926-7f5b-4a4c-8753-3eaa75be6ed4 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:44.752892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:44.768778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:44.769448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:44.769735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:44.770011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:44.770850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:44.771194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:44.772891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f216c-6cd1-4e28-910e-3c84d7c0de43 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:44.793618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 603/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:44.795479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:44.924448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0938-8be0-4f38-8d09-7e09e5a95211 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:44.956322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:44.963570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:44.964331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:44.964680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:44.965018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:44.965933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=702fa2f56dab462ea0c644ddb516ae8c) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:44.966314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:44.967885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbbe76-206f-4875-b46c-941fa1212301 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:44.979911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 616/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:07:44 2022] GET /identity/v3/role_assignments?user.id=702fa2f56dab462ea0c644ddb516ae8c&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:47.339752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566aa4f-0b93-4cb7-b60e-364617780717 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:47.340342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566aa4f-0b93-4cb7-b60e-364617780717 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:47.340342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d566aa4f-0b93-4cb7-b60e-364617780717 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:47.342449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 604/1220] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:07:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:07:47.348267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:47.348748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:47.349139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:47.370289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:07:47.372028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:07:47.461667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 617/1221] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:07:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:07:47.485599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566aa4f-0b93-4cb7-b60e-364617780717 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:47.603157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d566aa4f-0b93-4cb7-b60e-364617780717 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:47.645522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:47.646979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:47.647267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:47.647543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:47.661204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:07:47.669937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:47.671027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:47.677510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 605/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:07:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:47.678224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:47.812036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ad491-c754-43be-990a-65e6e571cb01 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:47.840324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:47.853810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:47.854426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:47.854715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:47.855045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:47.855962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] RBAC: Adding query filter params (name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:47.856342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:47.857863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:47.865986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 618/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:47.871853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:47.916478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:47.976308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc148657-f939-4d04-8633-90b9205d524f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:48.014159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:48.015179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:48.015567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:48.015878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:48.032183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:48.033879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:48.058365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:48.059917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:48.060818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:48.060818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:48.060818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:48.060818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:48.060818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 606/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:48.065506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:48.179141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e73db-bcae-429f-b81f-c34c98a5624e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:48.220285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:48.221034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:48.221339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:48.221644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:48.222432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:48.222776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:48.224884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:48.238364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 619/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:48.244598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:48.353020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:48.361743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87586d24-d9b0-498d-b646-d9df50b5a79b None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:48.398007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:48.398805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:48.399122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:48.399407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:48.407885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:48.409422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:07:48.423897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:48.425027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:48.426098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 607/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:48.431081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:48.483155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:48.545119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69b581-f789-4a5b-acf9-62722c917850 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:48.586135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:48.586910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:48.587200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:48.587477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:48.588408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:48.588766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:48.593820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:48.614122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 620/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Tue Jun 7 13:07:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:48.620079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:48.735515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff39e6a-d24f-4321-a0d0-6083235ef90c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:48.773395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:48.774258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:48.774690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:48.775017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:48.786704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:48.788194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:48.808292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:48.809521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:48.810598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:48.810598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:48.810598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:48.810598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:48.810598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 608/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:07:48.817095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:48.925166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9294272-528e-4263-a6e1-546319e74244 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:48.928419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:48.963497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:48.964158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:48.964442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:48.964717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:48.965438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:48.965776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:48.967181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:48.978526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 621/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:48.984076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:48.995109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:49.097876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf0541-0240-4bc3-a229-281e7132ad6d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:49.136258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:49.137134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:49.137417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:49.137727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:49.144627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:49.146098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:07:49.160855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:49.162460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:49.163580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 609/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:49.168527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:49.289061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0be6c2-39fe-4115-a1c7-7210fcf2e3e8 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:49.327767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:49.328837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:49.329161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:49.329624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:49.330644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:49.331032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:49.332439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e8821-7177-4e71-b083-c7d34be2b885 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:49.348672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 622/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:49.355300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:49.429437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:49.473859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ce246-1dfe-48b4-8cef-d6146e34431c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:49.515409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a23f9-94ff-43bf-bb63-93913a275b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:49.516331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a23f9-94ff-43bf-bb63-93913a275b1f None admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:49.516611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a23f9-94ff-43bf-bb63-93913a275b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:49.516902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a23f9-94ff-43bf-bb63-93913a275b1f None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/702fa2f56dab462ea0c644ddb516ae8c/roles/2fa0ed8c19534d9882a282faeff988e7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:49.534149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a23f9-94ff-43bf-bb63-93913a275b1f None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=702fa2f56dab462ea0c644ddb516ae8c, role_id=2fa0ed8c19534d9882a282faeff988e7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:49.535645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a23f9-94ff-43bf-bb63-93913a275b1f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:49.548401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 610/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:07:49 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/702fa2f56dab462ea0c644ddb516ae8c/roles/2fa0ed8c19534d9882a282faeff988e7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:07:51.959670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c07af0-cda8-457d-9a1f-9ec0122313fa None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:51.960284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c07af0-cda8-457d-9a1f-9ec0122313fa None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:51.960284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c07af0-cda8-457d-9a1f-9ec0122313fa None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:51.961575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 623/1233] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:07:51.967277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:51.967755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:51.967856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:51.987171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:07:51.988493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:07:52.070712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 611/1234] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:07:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:07:52.095227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c07af0-cda8-457d-9a1f-9ec0122313fa None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:52.121618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:52.212699 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c07af0-cda8-457d-9a1f-9ec0122313fa None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:52.252292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:52.253126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:52.253366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:52.253702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:52.269521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:07:52.281091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:52.282537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:52.283901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 624/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Tue Jun 7 13:07:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:52.288517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:52.341746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:52.404692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10c56f-7084-494c-b84c-e8582e275df0 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:52.442542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:52.443995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:52.444355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:52.444692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:52.445623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] RBAC: Adding query filter params (name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:52.446038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:52.447913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:52.457534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 612/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Tue Jun 7 13:07:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:07:52.463397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:52.558983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:52.579170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0c6c3-e32c-4cea-815d-95372dc4eef0 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:52.616264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:52.617090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:52.617336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:52.617611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:52.628774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:52.631142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:52.649304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:52.650443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:52.651430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:52.651430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:52.651430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:52.651430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:52.651430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 625/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:52.656756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:52.763373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751ea17-4dbd-4394-ba2a-1dc0476bd930 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:52.773877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:52.803045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:52.803718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:52.803996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:52.804290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:52.804995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:52.805301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:52.806681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:52.819701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 613/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:52.825886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:52.937096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b77134-6bc1-4986-9d9d-70b8a9473ec6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:52.976704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:52.977589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:52.977905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:52.978178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:52.981102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:52.987218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:52.989108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:07:53.004134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:53.005787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:53.006996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 626/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:07:53.012088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:53.138049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34111a73-77de-4c88-99eb-c5efb4c94a19 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:53.181446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:53.182162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:53.182476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:53.182749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:53.183716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:53.184090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:53.185644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:53.206604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 614/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Tue Jun 7 13:07:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:53.212808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:53.335692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad9d07-a69b-495d-ac97-32cb60c45384 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:53.370601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:53.371284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:53.371569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:53.371847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:53.436060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:53.443793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:53.445242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:53.555941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 627/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:07:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:07:53.562547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:53.593215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:53.672153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471adf4f-26e3-4148-91e1-a01b3918939c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:53.710408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:53.711165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:53.711457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:53.711732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:53.722717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:53.724260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:53.742731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:53.744700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:53.746076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:53.746076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:53.746076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:53.746076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:53.746076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 615/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:53.750019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:53.853496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:53.859701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4acd3-fd85-46fc-8276-46d1289f3490 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:53.897964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:53.898639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:53.898950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:53.899234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:53.899965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:53.900273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:53.901658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:53.915711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 628/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:53.921622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:54.007032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:54.029095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472d608-72b1-4d46-8a95-044f4dc02e47 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:54.069234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:54.070179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:54.070532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:54.070879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:54.078504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:54.080020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:07:54.093842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:54.095091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:54.096139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 616/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:54.101142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:54.208264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5cb3f-0526-4534-a426-41a8064b7604 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:54.253393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:54.254541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:54.255070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:54.255940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:54.257112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:54.257548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:54.259710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae420f1-c0a7-42ea-9514-c73324f8811f None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:54.279170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 629/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:54.285489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:54.404519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd00de5-25b3-459c-9a88-e0b10f157958 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:54.443360 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:54.444019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:54.444306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:54.444584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:54.445435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] RBAC: Adding query filter params (role.id=2fa0ed8c19534d9882a282faeff988e7, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=702fa2f56dab462ea0c644ddb516ae8c) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:54.445773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:54.447351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fcd3f-4a0b-4fb2-af0b-80adaf4b7330 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:54.453734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 617/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:07:54 2022] GET /identity/v3/role_assignments?user.id=702fa2f56dab462ea0c644ddb516ae8c&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=2fa0ed8c19534d9882a282faeff988e7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:56.863109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea76d6-641b-4e74-b564-09c7dc8a0aa2 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:56.863665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea76d6-641b-4e74-b564-09c7dc8a0aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:56.863665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea76d6-641b-4e74-b564-09c7dc8a0aa2 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:56.865872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 630/1247] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:07:56.871236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:56.871716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:56.871821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:56.876972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:56.889686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:07:56.891019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:07:56.963845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 618/1248] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:07:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 07 13:07:56.988354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea76d6-641b-4e74-b564-09c7dc8a0aa2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:57.008588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:57.102558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea76d6-641b-4e74-b564-09c7dc8a0aa2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:57.139879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:57.140634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:57.140912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:57.141214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:57.152662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:07:57.162931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:57.164383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:57.165257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 631/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:57.170639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:57.285730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691fcb4-436c-4a62-808c-dd19e4bc8f48 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:57.318814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:57.324121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:57.324959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:57.325243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:57.325515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:57.326262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:57.326565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:57.327974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:57.336947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 619/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:07:57.342881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:57.437647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:57.464434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e540f5e-3929-4a68-ad40-ee4a2bc31955 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:57.500561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:57.501369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:57.501636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:57.501911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:57.513883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:57.515375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:57.533293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:57.534348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:57.535296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:57.535296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:57.535296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:57.535296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:57.535296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 632/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:07:57.540767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:57.654998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06deaf-1413-462e-b209-baaf0c878e98 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:57.689792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:57.690372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:57.690654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:57.690962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:57.691703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:57.692008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:57.693984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:57.707606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 620/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:57.713537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:57.824085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe389ba9-b6b8-4174-926e-2478df4c0793 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:57.855672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:57.865175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:57.866081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:57.866357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:57.866614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:57.875091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:57.876554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:07:57.892074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:57.898415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:07:57.899739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 633/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:07:57.900407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:57.958800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:58.003560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47a20b-57fe-4253-ab09-100884c7e07f None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:58.039218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:58.039923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:58.040172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:58.040449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:58.041293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:58.041600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:58.043062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:58.063133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 621/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Tue Jun 7 13:07:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:58.070117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:58.185107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801960-951c-4fd4-8a99-66cae85d0324 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:58.224990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:58.225781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:58.226199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:58.226524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:58.309931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:58.326480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:58.328455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:58.459977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 634/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:07:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:07:58.461218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:58.578664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233f85-3880-45a2-a018-e3d8bcd2e239 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:58.593647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:58.616739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:58.617535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:58.617816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:58.618085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:58.628607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:58.630076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:07:58.650654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:58.651878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:58.653006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:07:58.653006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:07:58.653006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:07:58.653006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:58.653006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 622/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:07:58.659565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:58.761166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:58.775356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49266d-6873-4dcc-be6f-a4e05711a241 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:58.810996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:58.811674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:58.811956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:58.812230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:58.812970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:58.813278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:58.814703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:58.827861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 635/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:58.834235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:58.937232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac30f9-9a56-4be2-bef8-8ee79ae7c0dd None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:58.972844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:58.973645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:58.973929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:58.974203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:58.981740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:58.983279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:07:58.996926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:07:58.997830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:07:58.998757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 623/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 07 13:07:59.003959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:59.106134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd23fa-1825-4b71-820b-12e03c2b0302 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:59.139639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:59.140295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:59.140576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:59.140850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:59.141633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:59.141942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:59.143439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3bfc47-b28f-4f2a-8a53-98d8b628f5f4 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:59.155979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 636/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:07:59.161960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:07:59.189099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:07:59.277204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508b2b7-0604-4186-a2a4-ccaae7fd296d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:07:59.310816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:07:59.311478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:07:59.311760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:07:59.312032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:07:59.312862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=702fa2f56dab462ea0c644ddb516ae8c) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:07:59.313168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:07:59.314581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800017a-0771-495f-954f-75fc08777e68 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:07:59.320441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 624/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:07:59 2022] GET /identity/v3/role_assignments?user.id=702fa2f56dab462ea0c644ddb516ae8c&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:08:01.534105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9762-93c1-4682-9d6e-f37e50041520 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:01.534619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9762-93c1-4682-9d6e-f37e50041520 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:01.534737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f9762-93c1-4682-9d6e-f37e50041520 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:01.536055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 637/1261] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:08:01.541074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:01.541473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:01.541632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:01.558576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:08:01.559778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:08:01.627275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 625/1262] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:08:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:08:01.649799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9762-93c1-4682-9d6e-f37e50041520 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:01.661065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:01.753019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f9762-93c1-4682-9d6e-f37e50041520 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:01.787476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:01.788274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:01.788552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:01.788822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:01.798813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:08:01.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:01.807403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:01.808263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 638/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:01.813611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:01.833113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:01.912498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a134651-6e0a-4760-bb1a-53d51d5a6e63 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:01.945311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:01.945967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:01.946252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:01.946528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:01.947434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:01.947739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:01.949383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:01.957167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 626/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:01.963622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.027996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:02.060829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba44441-0176-4c92-a526-8c5b6a1e651c None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:02.093763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:02.094530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:02.094808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:02.095109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:02.105325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:02.106757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:08:02.123273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:02.124292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:02.125329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:08:02.125329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:08:02.125329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:02.125329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:02.125329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 639/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:02.130210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.209210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:02.234612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a291f2-5c77-4c10-b66f-618964739738 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:02.277709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:02.278495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:02.278728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:02.279039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:02.279766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:02.280071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:02.281663 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:02.293699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 627/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:02.299342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.396185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484004d-6a0e-467a-8721-c7102a6e248d None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:02.414871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:02.429605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:02.430398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:02.431133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:02.431133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:02.438315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:02.439792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:08:02.454407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:02.455348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:02.456330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 640/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:08:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:02.461351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.556097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a17851-825e-4599-bd3c-9dbfa763599d None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:02.589143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:02.589796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:02.590080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:02.590357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:02.591218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:02.591529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:02.592941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:02.601792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:02.610008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 628/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Tue Jun 7 13:08:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:02.616123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.719528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5574f-42ca-42e6-8e2f-0fea7c489330 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:02.751980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:02.752755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:02.753034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:02.753304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:02.770937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:02.772380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:08:02.789274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:02.790340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:02.791637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:08:02.791637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:08:02.791637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:02.791637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:02.791637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 641/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:02.795972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.892066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9105-c3e4-4e58-b26d-08868c845eff None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:02.924502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:02.925142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:02.925426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:02.925703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:02.926417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:02.926722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:02.928656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:02.939517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 629/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:02.944747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:02.976037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:03.042205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f41a9-816d-4c5f-b936-1fa3fd53af07 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:03.074040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:03.074810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:03.075123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:03.075399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:03.081822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:03.083277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:08:03.097536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:03.098560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:03.100150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 642/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:08:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:03.104817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:03.160849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:03.201269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d196e-4824-4522-95e5-02c24d4b5fa0 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:03.236485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:03.237135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:03.237421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:03.237765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:03.238551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:03.238937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:03.240552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ffb74-4c09-403b-b494-e2183298aa6b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:03.253276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 630/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:08:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:03.259187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:03.348565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:03.357722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18575a-5fec-4e75-8144-8d7320a342b6 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:03.390099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76f47-0e9c-4687-948a-ff89e451f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:03.391003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76f47-0e9c-4687-948a-ff89e451f9d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:03.391289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76f47-0e9c-4687-948a-ff89e451f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:03.391564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76f47-0e9c-4687-948a-ff89e451f9d8 None admin] PATH_INFO: `/v3/projects/34114a5ed9d147c585097748da3c008b/users/702fa2f56dab462ea0c644ddb516ae8c/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:03.393482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76f47-0e9c-4687-948a-ff89e451f9d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=34114a5ed9d147c585097748da3c008b, user_id=702fa2f56dab462ea0c644ddb516ae8c, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:03.395007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76f47-0e9c-4687-948a-ff89e451f9d8 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:03.407525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 643/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Tue Jun 7 13:08:03 2022] PUT /identity/v3/projects/34114a5ed9d147c585097748da3c008b/users/702fa2f56dab462ea0c644ddb516ae8c/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:08:05.598715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d182c4-8860-41ad-9aa7-d6ceb870ec86 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:05.599289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d182c4-8860-41ad-9aa7-d6ceb870ec86 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:05.599289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d182c4-8860-41ad-9aa7-d6ceb870ec86 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:05.600670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 631/1274] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:08:05.605868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:05.606286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:05.606423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:05.622629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:08:05.623848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:08:05.687706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 644/1275] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:08:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:08:05.712082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d182c4-8860-41ad-9aa7-d6ceb870ec86 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:05.809151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d182c4-8860-41ad-9aa7-d6ceb870ec86 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:05.826208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:05.842983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:05.843753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:05.844038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:05.844314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:05.854153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 07 13:08:05.861728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:05.862796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:05.863687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 632/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Tue Jun 7 13:08:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:05.868683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:05.947564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:05.965975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147bb4d-a1d8-4e16-b01a-88d12f2c9fa9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:05.998372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:05.999040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:05.999320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:05.999592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:06.000332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:06.000632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:06.001979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:06.010312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 645/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Tue Jun 7 13:08:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:06.015695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:06.111233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd0027-2a62-42c1-b87f-90d50c1a350e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:06.143733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:06.144497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:06.144782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:06.145053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:06.155155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:06.156584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:08:06.172921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:06.173962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:06.175212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:08:06.175212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:08:06.175212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:06.175212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:06.175212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 633/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:06.179999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:06.291371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e9766-5c11-4bf7-b45c-4409f5edec29 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:06.325003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:06.325655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:06.325946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:06.326222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:06.326973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:06.327284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:06.328649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:06.339987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 646/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:08:06.345599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:06.377141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:06.444681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc673d62-df72-4667-b268-e01dc79f498c None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:06.477552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:06.478346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:06.478633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:06.478939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:06.486184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:06.487652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 07 13:08:06.502293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:06.503293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:06.504293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 634/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:08:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:06.509184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:06.512941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:06.606734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430adbcb-2029-45ec-abb1-af7d30b27480 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:06.640436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:06.641099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:06.641383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:06.641658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:06.642499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:06.642811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:06.644268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:06.660936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 647/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Tue Jun 7 13:08:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:06.667116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:06.755029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:06.764236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd08891-1d01-4101-a46a-eff6322b0f31 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:06.796979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:06.797609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:06.797891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:06.798164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:06.862704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:06.864109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:06.947992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:06.958434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 635/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Tue Jun 7 13:08:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 07 13:08:06.964664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:07.024209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:07.061622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639cb3b-d78f-46b1-9891-1322efcb1d0e None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:07.095042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:07.095825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:07.096108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:07.096385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:07.106076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:07.107552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 07 13:08:07.124619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:07.125651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:07.126740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 07 13:08:07.126740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 07 13:08:07.126740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 07 13:08:07.126740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:07.126740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 648/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:07.131779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:07.227067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee697ab4-79f6-48e9-abf7-5e2b0d0e6c06 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:07.259694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:07.260333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:07.260617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:07.260893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:07.261609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:07.261912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:07.263308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:07.274757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 636/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Tue Jun 7 13:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:07.280361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:07.376835 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b997724-4eac-4dc6-a3fe-e0f90d81345a None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:07.391428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:07.411900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:07.412675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:07.412957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:07.413233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:07.419811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:07.421272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 07 13:08:07.435456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:07.436481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: ERROR keystone.server.flask.application Jun 07 13:08:07.437465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 649/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Tue Jun 7 13:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:07.442345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:07.491438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:07.539079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc22f68-82af-4de2-95d3-0000ccbc3b78 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:07.571720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:07.572359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:07.572641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:07.572914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:07.573957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:07.574263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:07.575712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda0f06-6508-46cc-8457-8b36e6a14b4b None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:07.587853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 637/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Tue Jun 7 13:08:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:07.593692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:07.689579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fceb3-e6ec-43ec-a1d6-78e584daf579 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:07.724822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:07.725471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:07.725756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:07.726034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:07.727124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] RBAC: Adding query filter params (role.id=4ad1dfee8ff946ffb420d546aad553ab, scope.project.id=34114a5ed9d147c585097748da3c008b, user.id=702fa2f56dab462ea0c644ddb516ae8c) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:07.727600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:07.729837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29648bc-2f62-4e24-b6aa-719851a64a59 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:07.737336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 650/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Tue Jun 7 13:08:07 2022] GET /identity/v3/role_assignments?user.id=702fa2f56dab462ea0c644ddb516ae8c&scope.project.id=34114a5ed9d147c585097748da3c008b&role.id=4ad1dfee8ff946ffb420d546aad553ab => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:09.953711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9bfa5-876e-41c3-8443-bb171f6bae36 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:09.954241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9bfa5-876e-41c3-8443-bb171f6bae36 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:09.954241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9bfa5-876e-41c3-8443-bb171f6bae36 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:09.955666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 638/1288] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:08:09.961501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:09.961876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:09.962050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:09.979076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:08:09.980269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:08:10.008472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:10.044981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 651/1289] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:08:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:08:10.071189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9bfa5-876e-41c3-8443-bb171f6bae36 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:10.171323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9bfa5-876e-41c3-8443-bb171f6bae36 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:10.181683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:10.204954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:10.205680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:10.205960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:10.206232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:10.206899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:10.208256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:10.221699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:10.223430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:10.224871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 639/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:08:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:10.229656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:10.335300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a469e5-3c54-441a-b5ac-5c2dfc19cdfb None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:10.367383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:10.368041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:10.368325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:10.368598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:10.369343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:10.369646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:10.371024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e539e2-f021-4ee4-94fb-eab8930bc801 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:10.375730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 652/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Tue Jun 7 13:08:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:10.382167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:10.480271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd714d1-110f-481c-82c0-02e246044a03 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:10.512705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:10.513350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:10.513632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:10.513903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] PATH_INFO: `/v3/services` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:10.514596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] RBAC: Adding query filter params (type=placement) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:10.514917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:10.516269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc7444-3275-4c10-b3c1-156c325126cf None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:10.520564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 640/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Tue Jun 7 13:08:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:08:12.696183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab1c98-2f75-4ca3-b163-09c5155391d9 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:12.696700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab1c98-2f75-4ca3-b163-09c5155391d9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:12.696700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab1c98-2f75-4ca3-b163-09c5155391d9 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:12.698020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 653/1293] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:08:12.703489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97a1bd-b6a8-4d03-a817-504f33401096 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:12.703907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97a1bd-b6a8-4d03-a817-504f33401096 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:12.704045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97a1bd-b6a8-4d03-a817-504f33401096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:12.721181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ee97a1bd-b6a8-4d03-a817-504f33401096 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:08:12.722363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ee97a1bd-b6a8-4d03-a817-504f33401096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:08:12.751489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:12.788957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 641/1294] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:08:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:08:12.811592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab1c98-2f75-4ca3-b163-09c5155391d9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:12.842886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:12.911352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab1c98-2f75-4ca3-b163-09c5155391d9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:12.944166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009d5d-3d13-4443-ae22-e183276850ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:12.944815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009d5d-3d13-4443-ae22-e183276850ff None admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:12.945099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009d5d-3d13-4443-ae22-e183276850ff None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:12.945380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009d5d-3d13-4443-ae22-e183276850ff None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:12.946090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009d5d-3d13-4443-ae22-e183276850ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:12.947530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009d5d-3d13-4443-ae22-e183276850ff None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:12.960393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 654/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Tue Jun 7 13:08:12 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:08:15.189149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f892ea2-9a21-4920-ade6-15b557e677f0 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:15.189712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f892ea2-9a21-4920-ade6-15b557e677f0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:15.189712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f892ea2-9a21-4920-ade6-15b557e677f0 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:15.191116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 642/1296] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:08:15.196406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:15.196813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:15.196960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:15.214259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:08:15.215685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:08:15.301398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 655/1297] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:08:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:08:15.324553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f892ea2-9a21-4920-ade6-15b557e677f0 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:15.423148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f892ea2-9a21-4920-ade6-15b557e677f0 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:15.467255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:15.467973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:15.468260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:15.468538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:15.469192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:15.470567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:15.482418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:15.483621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:15.484655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 643/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:08:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:15.489737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:15.507846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:15.585552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a4b43-a7a7-4d32-9106-8b1f1849da84 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:15.617739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:15.618374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:15.618656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:15.618954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:15.619648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:15.619946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:15.621289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:15.626217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 656/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Tue Jun 7 13:08:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:15.632440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:15.663364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:15.729079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c9b7-4453-48af-bfe4-3216bbb1b385 None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:15.761322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:15.761957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:15.762238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:15.762513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:15.763347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] RBAC: Adding query filter params (interface=public, service_id=9772a52347cc40638d76be1c1d717f0d, region_id=RegionOne) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:15.763652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:15.765025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ede17-2164-46e1-bc43-74fbd5ef1c05 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:15.769895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 644/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Tue Jun 7 13:08:15 2022] GET /identity/v3/endpoints?service_id=9772a52347cc40638d76be1c1d717f0d&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:15.775513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:15.853303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:15.871302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7787efc-f45e-4c11-bd1d-b9321d35a3c9 None admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:15.903906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ea535-79fe-461a-b6d6-4c39c9a21e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:15.904547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ea535-79fe-461a-b6d6-4c39c9a21e69 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:15.904832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ea535-79fe-461a-b6d6-4c39c9a21e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:15.905107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ea535-79fe-461a-b6d6-4c39c9a21e69 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:15.905777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ea535-79fe-461a-b6d6-4c39c9a21e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:15.907152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ea535-79fe-461a-b6d6-4c39c9a21e69 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:15.913681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 657/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Tue Jun 7 13:08:15 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:08:18.120465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632fccb-65f8-409c-af91-5c1962ea2a56 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:18.121023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632fccb-65f8-409c-af91-5c1962ea2a56 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:18.121023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632fccb-65f8-409c-af91-5c1962ea2a56 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:18.122384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 645/1302] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:08:18.127565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:18.127965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:18.128122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:18.144328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:08:18.145497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:08:18.209930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 658/1303] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:08:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 07 13:08:18.232928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632fccb-65f8-409c-af91-5c1962ea2a56 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:18.325686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:18.332614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632fccb-65f8-409c-af91-5c1962ea2a56 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:18.365484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:18.366265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:18.366553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:18.366847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:18.367507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:18.368867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 07 13:08:18.381287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 07 13:08:18.382357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.get_or_create( Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application with Lock( Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self._enter() Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application return self.creator() Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application created_value = creator( Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: ERROR keystone.server.flask.application Jun 07 13:08:18.383562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 646/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Tue Jun 7 13:08:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 07 13:08:18.388353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:18.464627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:18.486684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db79078-33c3-4eb9-b33b-f4ae826ab55a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:18.519189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:18.519842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:18.520126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:18.520400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] PATH_INFO: `/v3/services` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:18.521698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:08:18.522003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:18.523388 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2980c5-3d1a-4512-a7ed-806cb50e8a98 None admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:18.528360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 659/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Tue Jun 7 13:08:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:08:18.534029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:08:18.631056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fe6b5-75ed-48a7-864a-d8e667f7047e None admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:08:18.663418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4db638-3074-4151-a294-8006b0f09b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:08:18.664007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4db638-3074-4151-a294-8006b0f09b14 None admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:08:18.664294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4db638-3074-4151-a294-8006b0f09b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:08:18.664574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4db638-3074-4151-a294-8006b0f09b14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:08:18.665314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4db638-3074-4151-a294-8006b0f09b14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:08:18.666673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4db638-3074-4151-a294-8006b0f09b14 None admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:08:18.684807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 647/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Tue Jun 7 13:08:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:08:28.525384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=30 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:08:28.680784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:22.111580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7d5f-711e-4d08-8b85-9048a3160735 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:22.112066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7d5f-711e-4d08-8b85-9048a3160735 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:22.112146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc7d5f-711e-4d08-8b85-9048a3160735 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:22.119153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ab7f-5930-459e-8664-004237e0c7d4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:22.119580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ab7f-5930-459e-8664-004237e0c7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:22.119749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724ab7f-5930-459e-8664-004237e0c7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:22.121883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 660/1307] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:22 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:22.209557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3724ab7f-5930-459e-8664-004237e0c7d4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:22.210954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3724ab7f-5930-459e-8664-004237e0c7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:22.347197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 648/1308] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:09:22.382515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c726-7e32-4894-935c-7937a1c6baab None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:22.382905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c726-7e32-4894-935c-7937a1c6baab None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:22.383147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3c726-7e32-4894-935c-7937a1c6baab None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:22.384821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 661/1309] 10.222.0.61 () {54 vars in 964 bytes} [Tue Jun 7 13:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:22.390756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03803dd-3e7f-49e3-b287-64e78d1957d4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:22.391146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03803dd-3e7f-49e3-b287-64e78d1957d4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:22.391385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03803dd-3e7f-49e3-b287-64e78d1957d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:22.486522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f03803dd-3e7f-49e3-b287-64e78d1957d4 None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:22.487772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f03803dd-3e7f-49e3-b287-64e78d1957d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:22.511709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:22.571142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 649/1310] 10.222.0.61 () {58 vars in 1061 bytes} [Tue Jun 7 13:09:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:22.581068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c726-7e32-4894-935c-7937a1c6baab None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:22.582940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-29c3c726-7e32-4894-935c-7937a1c6baab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:22.696073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3c726-7e32-4894-935c-7937a1c6baab None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:22.780766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:22.781421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:22.781707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:22.781985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:22.783682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:22.841596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:22.901150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:22.902591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496c7f7-1566-463a-a7c8-f51649c9b53e service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:22.921867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 662/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:25.485635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8edc6-6def-417e-8774-288bd924b50c None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:25.486650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8edc6-6def-417e-8774-288bd924b50c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:25.487076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8edc6-6def-417e-8774-288bd924b50c None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:25.488600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 650/1312] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:25.494883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c95c-8d09-47b9-88ca-f6898b326f96 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:25.495343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c95c-8d09-47b9-88ca-f6898b326f96 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:25.495628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c95c-8d09-47b9-88ca-f6898b326f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:25.568831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e2f3c95c-8d09-47b9-88ca-f6898b326f96 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:25.570218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e2f3c95c-8d09-47b9-88ca-f6898b326f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:25.671991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 663/1313] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:25.710715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8edc6-6def-417e-8774-288bd924b50c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:25.712297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a7e8edc6-6def-417e-8774-288bd924b50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:25.819344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8edc6-6def-417e-8774-288bd924b50c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:25.897890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:25.899866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:25.900934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:25.901284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:25.901627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:25.904063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:26.152356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:26.153716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4c7d7-58fb-4b40-a37f-5291f1bf62ab service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:26.172903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 651/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:28.795931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9ec54-840b-43f7-98ab-0b32fbaf734c None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:28.796491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9ec54-840b-43f7-98ab-0b32fbaf734c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:28.796491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9ec54-840b-43f7-98ab-0b32fbaf734c None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:28.798080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 664/1315] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:28.803992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3128d-e58f-45f4-99f2-437a9ca409af None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:28.804512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3128d-e58f-45f4-99f2-437a9ca409af None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:28.804601 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3128d-e58f-45f4-99f2-437a9ca409af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:28.871107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-eab3128d-e58f-45f4-99f2-437a9ca409af None None] MFA Rules not processed for user `1d6906ac8c4644eeb5bc8ca0a350e99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:28.872332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-eab3128d-e58f-45f4-99f2-437a9ca409af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:28.922976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:28.967675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 652/1316] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:29.007627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9ec54-840b-43f7-98ab-0b32fbaf734c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:29.009161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-49d9ec54-840b-43f7-98ab-0b32fbaf734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:29.018695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:29.111235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9ec54-840b-43f7-98ab-0b32fbaf734c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:29.190422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:29.191085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:29.191365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:29.191640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:29.193329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:29.253921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:29.300436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:29.301865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683e16-de83-4911-9a77-73553e395fc1 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:29.320771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 665/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:32.112736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169a173-568b-4104-9861-a1df7990c224 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:32.113259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169a173-568b-4104-9861-a1df7990c224 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:32.113374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169a173-568b-4104-9861-a1df7990c224 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:32.114655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 653/1318] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:32.120545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34b5f-ecc5-4ff6-ab5a-c0659d1107ca None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:32.120953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34b5f-ecc5-4ff6-ab5a-c0659d1107ca None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:32.121095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb34b5f-ecc5-4ff6-ab5a-c0659d1107ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:32.183084 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2cb34b5f-ecc5-4ff6-ab5a-c0659d1107ca None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:32.184267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2cb34b5f-ecc5-4ff6-ab5a-c0659d1107ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:32.278181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 666/1319] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:32.315993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169a173-568b-4104-9861-a1df7990c224 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:32.317558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d169a173-568b-4104-9861-a1df7990c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:32.419706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169a173-568b-4104-9861-a1df7990c224 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:32.491257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:32.491897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:32.492180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:32.492456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:32.494094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:32.500841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:32.601456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:32.602928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d8447-6edf-40b6-9b20-5b1d336554e2 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:32.621147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 654/1320] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:35.537356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d39996-c1b9-46d0-93c0-8840b8e90900 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:35.537910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d39996-c1b9-46d0-93c0-8840b8e90900 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:35.538016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d39996-c1b9-46d0-93c0-8840b8e90900 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:35.539382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 667/1321] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:35.544917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340a75f-b539-4aba-9107-294388c1da0b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:35.545356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340a75f-b539-4aba-9107-294388c1da0b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:35.545453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340a75f-b539-4aba-9107-294388c1da0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:35.611359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8340a75f-b539-4aba-9107-294388c1da0b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:35.612543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8340a75f-b539-4aba-9107-294388c1da0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:35.670714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:35.710921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 655/1322] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:35.748491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d39996-c1b9-46d0-93c0-8840b8e90900 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:35.750133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-44d39996-c1b9-46d0-93c0-8840b8e90900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:35.769395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:35.854804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d39996-c1b9-46d0-93c0-8840b8e90900 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:35.929628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:35.930280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:35.930562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:35.930863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:35.932562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:36.009197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:36.046674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:36.048140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dea615-e04f-4b3b-82ad-7dd52d12a22b service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:36.066778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 668/1323] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:38.381423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f9c0c-57fd-4a1b-a843-3152816a93b5 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:38.381934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f9c0c-57fd-4a1b-a843-3152816a93b5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:38.382227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f9c0c-57fd-4a1b-a843-3152816a93b5 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:38.383678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 656/1324] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:38.388917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fbcb5-6c66-4deb-bfa9-6a56a0d657c4 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:38.389345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fbcb5-6c66-4deb-bfa9-6a56a0d657c4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:38.389470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fbcb5-6c66-4deb-bfa9-6a56a0d657c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:38.458101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0a2fbcb5-6c66-4deb-bfa9-6a56a0d657c4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:38.459818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0a2fbcb5-6c66-4deb-bfa9-6a56a0d657c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:38.557325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 669/1325] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:38.593338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f9c0c-57fd-4a1b-a843-3152816a93b5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:38.594930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c56f9c0c-57fd-4a1b-a843-3152816a93b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:38.695418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f9c0c-57fd-4a1b-a843-3152816a93b5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:38.766597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:38.767264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:38.767551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:38.767825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:38.769454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:38.786206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:38.876166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:38.877601 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335bc56c-3508-472a-ab33-47aab95a95d4 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:38.896086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 657/1326] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:41.193983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bcd9a-1d86-47ed-97be-83afc13159eb None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:41.194547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bcd9a-1d86-47ed-97be-83afc13159eb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:41.194547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305bcd9a-1d86-47ed-97be-83afc13159eb None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:41.195928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 670/1327] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:41.201131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371614fc-1813-44f0-8943-8a2caea096a7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:41.201568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371614fc-1813-44f0-8943-8a2caea096a7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:41.201816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371614fc-1813-44f0-8943-8a2caea096a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:41.272595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-371614fc-1813-44f0-8943-8a2caea096a7 None None] MFA Rules not processed for user `1d6906ac8c4644eeb5bc8ca0a350e99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:41.273797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-371614fc-1813-44f0-8943-8a2caea096a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:41.339390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:41.372094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 658/1328] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:41.424565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b40c85-93f5-4b28-9e4f-02de42a69264 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:41.424852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b40c85-93f5-4b28-9e4f-02de42a69264 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:41.425129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b40c85-93f5-4b28-9e4f-02de42a69264 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:41.429328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 671/1329] 10.222.0.61 () {54 vars in 964 bytes} [Tue Jun 7 13:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:41.433202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dbd87-f681-44b1-8828-4fc3f2b8e8bb None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:41.433499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dbd87-f681-44b1-8828-4fc3f2b8e8bb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:41.433777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05dbd87-f681-44b1-8828-4fc3f2b8e8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:41.507663 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d05dbd87-f681-44b1-8828-4fc3f2b8e8bb None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:41.508850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d05dbd87-f681-44b1-8828-4fc3f2b8e8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:41.584071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 659/1330] 10.222.0.61 () {58 vars in 1061 bytes} [Tue Jun 7 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:41.593039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b40c85-93f5-4b28-9e4f-02de42a69264 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:41.594671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-32b40c85-93f5-4b28-9e4f-02de42a69264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:41.625326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:41.696940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b40c85-93f5-4b28-9e4f-02de42a69264 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:41.770171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:41.770742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:41.771049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:41.771324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:41.772994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:41.847116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:41.879030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:41.880452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa72ab7-d758-4844-b46e-2f8e761b9c8b service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:41.897945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 672/1331] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:44.773546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f729b7-2943-4e97-b524-22966a221378 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:44.774104 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f729b7-2943-4e97-b524-22966a221378 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:44.774104 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f729b7-2943-4e97-b524-22966a221378 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:44.775502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 660/1332] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:44.780923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0749-7e88-487b-851f-9748d98e6509 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:44.781382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0749-7e88-487b-851f-9748d98e6509 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:44.781500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb0749-7e88-487b-851f-9748d98e6509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:44.848933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d5bb0749-7e88-487b-851f-9748d98e6509 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:44.850685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d5bb0749-7e88-487b-851f-9748d98e6509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:44.950766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 673/1333] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:44.986923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f729b7-2943-4e97-b524-22966a221378 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:44.988444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-96f729b7-2943-4e97-b524-22966a221378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:45.057296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:45.092029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f729b7-2943-4e97-b524-22966a221378 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:45.164667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:45.165242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:45.165526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:45.165798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:45.167475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:45.276173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:45.277607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a947-a05a-4e45-bbbf-8c406b1707a8 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:45.288028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:45.296514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 661/1334] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:47.569918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7e9ab-bbce-4b21-86a4-20e206b34d7e None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:47.570446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7e9ab-bbce-4b21-86a4-20e206b34d7e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:47.570446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7e9ab-bbce-4b21-86a4-20e206b34d7e None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:47.571873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 674/1335] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:47.577441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558f2a4-29e8-4307-8e93-f5dbb6985da8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:47.577864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558f2a4-29e8-4307-8e93-f5dbb6985da8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:47.577998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558f2a4-29e8-4307-8e93-f5dbb6985da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:47.640902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a558f2a4-29e8-4307-8e93-f5dbb6985da8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:47.642099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a558f2a4-29e8-4307-8e93-f5dbb6985da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:47.736201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 662/1336] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:47.775538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7e9ab-bbce-4b21-86a4-20e206b34d7e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:47.777099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7ad7e9ab-bbce-4b21-86a4-20e206b34d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:47.943151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:48.005366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7e9ab-bbce-4b21-86a4-20e206b34d7e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:48.076154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:48.076791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:48.077070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:48.077607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:48.079731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:48.164850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:48.190981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:48.192414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721ceb3-3de8-4742-a59f-8789b34bf6eb service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:48.210873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 675/1337] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:50.755412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5674d47-d93d-4709-a2e6-64b92c2034b1 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:50.756422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5674d47-d93d-4709-a2e6-64b92c2034b1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:50.756516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5674d47-d93d-4709-a2e6-64b92c2034b1 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:50.757863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 663/1338] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:50.763152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2476ec-2d43-4963-88d5-bf43e5c0f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:50.763572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2476ec-2d43-4963-88d5-bf43e5c0f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:50.763730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2476ec-2d43-4963-88d5-bf43e5c0f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:50.826174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-bf2476ec-2d43-4963-88d5-bf43e5c0f8a8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:50.827655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bf2476ec-2d43-4963-88d5-bf43e5c0f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:50.922311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 676/1339] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:50.958619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5674d47-d93d-4709-a2e6-64b92c2034b1 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:50.960184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b5674d47-d93d-4709-a2e6-64b92c2034b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:51.033014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:51.060201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5674d47-d93d-4709-a2e6-64b92c2034b1 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:51.131570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:51.132203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:51.132487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:51.132750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:51.134366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:51.240043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:51.241858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995760c2-a190-47d6-9441-fdbdaae609d3 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:51.261235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 664/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:55.580873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596004ea-a7ce-4fe6-9626-0047bca4605f None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:55.581440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596004ea-a7ce-4fe6-9626-0047bca4605f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:55.581440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596004ea-a7ce-4fe6-9626-0047bca4605f None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:55.582728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 677/1341] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:55.588162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfefc27-53c2-47f9-aaa3-303a43efa572 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:55.588607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfefc27-53c2-47f9-aaa3-303a43efa572 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:55.588703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfefc27-53c2-47f9-aaa3-303a43efa572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:55.594803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:55.659720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8bfefc27-53c2-47f9-aaa3-303a43efa572 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:55.660911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8bfefc27-53c2-47f9-aaa3-303a43efa572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:55.765630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 665/1342] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:55.803971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596004ea-a7ce-4fe6-9626-0047bca4605f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:55.806228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-596004ea-a7ce-4fe6-9626-0047bca4605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:55.857535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:09:55.903189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596004ea-a7ce-4fe6-9626-0047bca4605f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:55.906984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6b0c3-4525-45a1-8ed6-e0e582771ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:55.907774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b0c3-4525-45a1-8ed6-e0e582771ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:55.908127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b0c3-4525-45a1-8ed6-e0e582771ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:55.908484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b0c3-4525-45a1-8ed6-e0e582771ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:55.918540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6b0c3-4525-45a1-8ed6-e0e582771ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:55.920131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6b0c3-4525-45a1-8ed6-e0e582771ca0 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:55.930251 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 678/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:09:58.775691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307db315-9909-4bae-a218-b7f943b7d4f4 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:58.776269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307db315-9909-4bae-a218-b7f943b7d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:58.776269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307db315-9909-4bae-a218-b7f943b7d4f4 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:58.777787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 666/1344] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:09:58.782895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a37d-bc9b-4c22-9dc4-78f4ec68be15 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:58.783300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a37d-bc9b-4c22-9dc4-78f4ec68be15 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:58.783459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977a37d-bc9b-4c22-9dc4-78f4ec68be15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:58.811031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b977a37d-bc9b-4c22-9dc4-78f4ec68be15 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:09:58.812219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b977a37d-bc9b-4c22-9dc4-78f4ec68be15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:09:58.888438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 679/1345] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:09:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:09:58.925654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307db315-9909-4bae-a218-b7f943b7d4f4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:09:58.937055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307db315-9909-4bae-a218-b7f943b7d4f4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:09:58.940582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65fe53-16b5-4931-92b0-f1f43d90a60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:09:58.941222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65fe53-16b5-4931-92b0-f1f43d90a60f service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:09:58.941506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65fe53-16b5-4931-92b0-f1f43d90a60f service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:09:58.941785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65fe53-16b5-4931-92b0-f1f43d90a60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:09:58.950791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65fe53-16b5-4931-92b0-f1f43d90a60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:09:58.952247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65fe53-16b5-4931-92b0-f1f43d90a60f service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:09:58.961886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 667/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:09:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:03.084099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ef724-d2e1-4ae7-b42d-48e9b4e0a109 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:03.084660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ef724-d2e1-4ae7-b42d-48e9b4e0a109 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:03.084660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ef724-d2e1-4ae7-b42d-48e9b4e0a109 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:03.085997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 680/1347] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:03.091350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85ec6c-4bd4-481d-9c7a-8d0b6ffbee58 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:03.091780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85ec6c-4bd4-481d-9c7a-8d0b6ffbee58 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:03.091922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85ec6c-4bd4-481d-9c7a-8d0b6ffbee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:03.110115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6f85ec6c-4bd4-481d-9c7a-8d0b6ffbee58 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:03.111337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6f85ec6c-4bd4-481d-9c7a-8d0b6ffbee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:03.119706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 668/1348] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:03.157233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ef724-d2e1-4ae7-b42d-48e9b4e0a109 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:03.167733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ef724-d2e1-4ae7-b42d-48e9b4e0a109 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:03.171215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cd81b-f548-4e77-b611-437ebf5b78db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:03.171847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cd81b-f548-4e77-b611-437ebf5b78db service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:03.172128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cd81b-f548-4e77-b611-437ebf5b78db service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:03.172403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cd81b-f548-4e77-b611-437ebf5b78db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:03.181676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cd81b-f548-4e77-b611-437ebf5b78db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:03.183139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cd81b-f548-4e77-b611-437ebf5b78db service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:03.192462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 681/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:08.585518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff6eed-9012-4a1a-9f28-761acfaf231a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:08.586416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff6eed-9012-4a1a-9f28-761acfaf231a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:08.587025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff6eed-9012-4a1a-9f28-761acfaf231a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:08.589397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 669/1350] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:08.594292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763366d8-06ae-4570-987b-70854a59bacc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:08.595115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763366d8-06ae-4570-987b-70854a59bacc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:08.595568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763366d8-06ae-4570-987b-70854a59bacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:08.668308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-763366d8-06ae-4570-987b-70854a59bacc None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:08.669411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-763366d8-06ae-4570-987b-70854a59bacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:08.697153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:10:08.747130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 682/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:13.063753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ad8b-6253-424f-9800-17d59fcadfa6 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:13.064316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ad8b-6253-424f-9800-17d59fcadfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:13.064316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37ad8b-6253-424f-9800-17d59fcadfa6 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:13.066040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 670/1352] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:13.071422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99827569-ae83-4634-aa93-c17647ff812c None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:13.071890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99827569-ae83-4634-aa93-c17647ff812c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:13.071984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99827569-ae83-4634-aa93-c17647ff812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:13.090593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-99827569-ae83-4634-aa93-c17647ff812c None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:13.091833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-99827569-ae83-4634-aa93-c17647ff812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:13.101162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 683/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:10:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:13.104818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=72 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:10:14.933858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0a00-c242-4c37-829c-6d523d8c596e None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:14.934499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0a00-c242-4c37-829c-6d523d8c596e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:14.934499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0a00-c242-4c37-829c-6d523d8c596e None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:14.935901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 671/1354] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:14.940900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f294c8-0c61-4769-b35c-ab4254b83813 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:14.941300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f294c8-0c61-4769-b35c-ab4254b83813 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:14.941449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f294c8-0c61-4769-b35c-ab4254b83813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:14.959907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d4f294c8-0c61-4769-b35c-ab4254b83813 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:14.961081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d4f294c8-0c61-4769-b35c-ab4254b83813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:14.969577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 684/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:18.560327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144e39e-ce6f-44ec-bbcd-edea3bbcd9c8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:18.560852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144e39e-ce6f-44ec-bbcd-edea3bbcd9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:18.560852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144e39e-ce6f-44ec-bbcd-edea3bbcd9c8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:18.562221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 672/1356] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:18.567438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132865-d828-432a-bd93-d4efda1e8e23 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:18.567844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132865-d828-432a-bd93-d4efda1e8e23 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:18.567987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e132865-d828-432a-bd93-d4efda1e8e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:18.594350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9e132865-d828-432a-bd93-d4efda1e8e23 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:18.596180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9e132865-d828-432a-bd93-d4efda1e8e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:18.605018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 685/1357] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:18.661851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b575-1781-4a65-9751-2e74d3b53770 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:18.662869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b575-1781-4a65-9751-2e74d3b53770 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:18.663209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985b575-1781-4a65-9751-2e74d3b53770 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:18.664965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 673/1358] 10.222.0.61 () {54 vars in 961 bytes} [Tue Jun 7 13:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:18.673530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2057cb0-ba3a-46d1-81cc-adb8c3e89466 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:18.674024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2057cb0-ba3a-46d1-81cc-adb8c3e89466 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:18.674397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2057cb0-ba3a-46d1-81cc-adb8c3e89466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:18.728028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a2057cb0-ba3a-46d1-81cc-adb8c3e89466 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:18.729376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a2057cb0-ba3a-46d1-81cc-adb8c3e89466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:18.815546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 686/1359] 10.222.0.61 () {58 vars in 1058 bytes} [Tue Jun 7 13:10:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:18.823843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985b575-1781-4a65-9751-2e74d3b53770 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:18.839599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985b575-1781-4a65-9751-2e74d3b53770 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:18.845203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc51cda-c5c7-4a7a-94b7-2b28fb895684 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:18.845839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc51cda-c5c7-4a7a-94b7-2b28fb895684 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:18.846121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc51cda-c5c7-4a7a-94b7-2b28fb895684 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:18.846393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc51cda-c5c7-4a7a-94b7-2b28fb895684 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:18.856259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc51cda-c5c7-4a7a-94b7-2b28fb895684 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:18.858058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc51cda-c5c7-4a7a-94b7-2b28fb895684 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:18.868540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 674/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:21.347751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e169ad5-2b38-476a-a0f2-9c5667f8f30d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:21.348318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e169ad5-2b38-476a-a0f2-9c5667f8f30d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:21.348318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e169ad5-2b38-476a-a0f2-9c5667f8f30d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:21.350332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 687/1361] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:21.356093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a5c2f-9435-45a6-9d30-bcb8175a9abd None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:21.356723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a5c2f-9435-45a6-9d30-bcb8175a9abd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:21.357005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a5c2f-9435-45a6-9d30-bcb8175a9abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:21.376831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8a2a5c2f-9435-45a6-9d30-bcb8175a9abd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:21.378022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8a2a5c2f-9435-45a6-9d30-bcb8175a9abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:21.386941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 675/1362] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:21.434754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e169ad5-2b38-476a-a0f2-9c5667f8f30d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:21.449469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e169ad5-2b38-476a-a0f2-9c5667f8f30d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:21.455361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167e1b3-24f1-4b84-b3c9-bd77e9b0f8e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:21.456110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167e1b3-24f1-4b84-b3c9-bd77e9b0f8e8 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:21.456498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167e1b3-24f1-4b84-b3c9-bd77e9b0f8e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:21.456876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167e1b3-24f1-4b84-b3c9-bd77e9b0f8e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:21.466501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167e1b3-24f1-4b84-b3c9-bd77e9b0f8e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:21.468266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167e1b3-24f1-4b84-b3c9-bd77e9b0f8e8 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:21.479828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 688/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:24.077946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0675159-4c4b-4d2c-baf0-cdce41c1967b None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:24.079132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0675159-4c4b-4d2c-baf0-cdce41c1967b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:24.079572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0675159-4c4b-4d2c-baf0-cdce41c1967b None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:24.081109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 676/1364] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:24.087294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142bf6-f57f-4769-b7ae-5904fddfe80a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:24.088082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142bf6-f57f-4769-b7ae-5904fddfe80a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:24.088514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7142bf6-f57f-4769-b7ae-5904fddfe80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:24.106949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f7142bf6-f57f-4769-b7ae-5904fddfe80a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:24.108184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f7142bf6-f57f-4769-b7ae-5904fddfe80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:24.115982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 689/1365] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:24.160376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6f1be-b8bb-48aa-9b7b-8bf12609dee0 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:24.160667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6f1be-b8bb-48aa-9b7b-8bf12609dee0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:24.160949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6f1be-b8bb-48aa-9b7b-8bf12609dee0 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:24.162997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 677/1366] 10.222.0.61 () {54 vars in 961 bytes} [Tue Jun 7 13:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:24.168227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f642f-f7de-441e-a7e9-7caa972715dc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:24.168515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f642f-f7de-441e-a7e9-7caa972715dc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:24.168789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f642f-f7de-441e-a7e9-7caa972715dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:24.186149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-c53f642f-f7de-441e-a7e9-7caa972715dc None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:24.187343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c53f642f-f7de-441e-a7e9-7caa972715dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:24.196456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 690/1367] 10.222.0.61 () {58 vars in 1058 bytes} [Tue Jun 7 13:10:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:24.204067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6f1be-b8bb-48aa-9b7b-8bf12609dee0 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:24.218737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6f1be-b8bb-48aa-9b7b-8bf12609dee0 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:24.224138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b692673-ff3c-4746-aa56-3d0d7b3b7be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:24.225392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b692673-ff3c-4746-aa56-3d0d7b3b7be0 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:24.225669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b692673-ff3c-4746-aa56-3d0d7b3b7be0 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:24.225949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b692673-ff3c-4746-aa56-3d0d7b3b7be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:24.234559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b692673-ff3c-4746-aa56-3d0d7b3b7be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:24.236161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b692673-ff3c-4746-aa56-3d0d7b3b7be0 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:24.245558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 678/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:10:26.563383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e0787-0880-46e3-90b6-f732f58e8ed6 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:26.564283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e0787-0880-46e3-90b6-f732f58e8ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:26.564722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e0787-0880-46e3-90b6-f732f58e8ed6 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:26.566248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 691/1369] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:26.572303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e4bb-7d96-4dbb-9e62-ea1d6543b3c9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:26.572800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e4bb-7d96-4dbb-9e62-ea1d6543b3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:26.573070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e4bb-7d96-4dbb-9e62-ea1d6543b3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:26.590993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bba8e4bb-7d96-4dbb-9e62-ea1d6543b3c9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:26.592312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bba8e4bb-7d96-4dbb-9e62-ea1d6543b3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:26.600903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 679/1370] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:26.645711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e0787-0880-46e3-90b6-f732f58e8ed6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:26.660910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e0787-0880-46e3-90b6-f732f58e8ed6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:26.667342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafa5c3-fab8-483c-9c6c-790f6e989091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:26.668102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafa5c3-fab8-483c-9c6c-790f6e989091 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:26.668476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafa5c3-fab8-483c-9c6c-790f6e989091 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:26.668866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafa5c3-fab8-483c-9c6c-790f6e989091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:26.677750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafa5c3-fab8-483c-9c6c-790f6e989091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:26.680041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafa5c3-fab8-483c-9c6c-790f6e989091 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:26.690269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 692/1371] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:26.801034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5c5be-2f9c-4b37-8f1b-8de1375b0c9a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:26.801326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5c5be-2f9c-4b37-8f1b-8de1375b0c9a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:26.801818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5c5be-2f9c-4b37-8f1b-8de1375b0c9a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:26.804027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 680/1372] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:26.810125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8209f6-5297-4f01-86f4-546819cf4d99 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:26.810687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8209f6-5297-4f01-86f4-546819cf4d99 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:26.811158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8209f6-5297-4f01-86f4-546819cf4d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:26.832876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ca8209f6-5297-4f01-86f4-546819cf4d99 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:26.834278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ca8209f6-5297-4f01-86f4-546819cf4d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:26.846875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 693/1373] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:27.153824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab5f3f-bb7c-4af6-9e35-7c27eb62c28a None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:27.153824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab5f3f-bb7c-4af6-9e35-7c27eb62c28a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:27.153824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab5f3f-bb7c-4af6-9e35-7c27eb62c28a None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:27.154508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 681/1374] 10.222.0.61 () {54 vars in 963 bytes} [Tue Jun 7 13:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:27.159303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10314e3-ae47-4948-89ca-d3027b6aa080 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:27.159590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10314e3-ae47-4948-89ca-d3027b6aa080 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:27.159866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10314e3-ae47-4948-89ca-d3027b6aa080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:27.166977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:10:27.210274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e10314e3-ae47-4948-89ca-d3027b6aa080 None None] MFA Rules not processed for user `27c81d5738744687a915940977f0f647`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:27.211503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e10314e3-ae47-4948-89ca-d3027b6aa080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:27.276493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 694/1375] 10.222.0.61 () {58 vars in 1060 bytes} [Tue Jun 7 13:10:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:27.284109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab5f3f-bb7c-4af6-9e35-7c27eb62c28a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:27.294754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab5f3f-bb7c-4af6-9e35-7c27eb62c28a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:27.298291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a484118-bd92-4895-8e37-0d45197582d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:27.298963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a484118-bd92-4895-8e37-0d45197582d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:27.299249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a484118-bd92-4895-8e37-0d45197582d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:27.299536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a484118-bd92-4895-8e37-0d45197582d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:27.309282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a484118-bd92-4895-8e37-0d45197582d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:27.310719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a484118-bd92-4895-8e37-0d45197582d8 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:27.321109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 682/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:28.989975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3607f-d2c4-4dc2-91a0-18b3afa5b2d9 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:28.991118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3607f-d2c4-4dc2-91a0-18b3afa5b2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:28.991231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3607f-d2c4-4dc2-91a0-18b3afa5b2d9 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:28.992669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 695/1377] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:28.998729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cef4d-8214-4176-91b2-c301293d8f2c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:28.999322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cef4d-8214-4176-91b2-c301293d8f2c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:28.999322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086cef4d-8214-4176-91b2-c301293d8f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:29.018014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-086cef4d-8214-4176-91b2-c301293d8f2c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:29.019314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-086cef4d-8214-4176-91b2-c301293d8f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:29.027862 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 683/1378] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:29.073647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3607f-d2c4-4dc2-91a0-18b3afa5b2d9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:29.089472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3607f-d2c4-4dc2-91a0-18b3afa5b2d9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:29.095197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d288417-507e-41f8-a045-82cb044a3d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:29.095870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d288417-507e-41f8-a045-82cb044a3d7d service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:29.096159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d288417-507e-41f8-a045-82cb044a3d7d service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:29.096439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d288417-507e-41f8-a045-82cb044a3d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:29.106262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d288417-507e-41f8-a045-82cb044a3d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:29.107838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d288417-507e-41f8-a045-82cb044a3d7d service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:29.119006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 696/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:31.404324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528075c4-2b7d-4049-b3ee-bd9a192af738 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:31.404923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528075c4-2b7d-4049-b3ee-bd9a192af738 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:31.404923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528075c4-2b7d-4049-b3ee-bd9a192af738 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:31.406242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 684/1380] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:31.411667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326fb7-e5e8-4f83-b811-d922211f7bfd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:31.412118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326fb7-e5e8-4f83-b811-d922211f7bfd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:31.412259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326fb7-e5e8-4f83-b811-d922211f7bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:31.431043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ac326fb7-e5e8-4f83-b811-d922211f7bfd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:31.433083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ac326fb7-e5e8-4f83-b811-d922211f7bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:31.441285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 697/1381] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:31.485530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528075c4-2b7d-4049-b3ee-bd9a192af738 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:31.500941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528075c4-2b7d-4049-b3ee-bd9a192af738 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:31.507212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b00c6a-4f93-47e1-b06a-5d3093e05051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:31.507879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b00c6a-4f93-47e1-b06a-5d3093e05051 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:31.508163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b00c6a-4f93-47e1-b06a-5d3093e05051 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:31.508439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b00c6a-4f93-47e1-b06a-5d3093e05051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:31.517101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b00c6a-4f93-47e1-b06a-5d3093e05051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:31.518674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b00c6a-4f93-47e1-b06a-5d3093e05051 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:31.528208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 685/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:33.814130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392407b-4ea4-485f-b48a-1d8da84e37a4 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:33.814730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392407b-4ea4-485f-b48a-1d8da84e37a4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:33.814730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9392407b-4ea4-485f-b48a-1d8da84e37a4 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:33.816206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 698/1383] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:33.822755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efd17-6138-4286-ae9f-9a424767cdb7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:33.823340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efd17-6138-4286-ae9f-9a424767cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:33.823686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29efd17-6138-4286-ae9f-9a424767cdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:33.844269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e29efd17-6138-4286-ae9f-9a424767cdb7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:33.845500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e29efd17-6138-4286-ae9f-9a424767cdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:33.858274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 686/1384] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:33.899375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392407b-4ea4-485f-b48a-1d8da84e37a4 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:33.914700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9392407b-4ea4-485f-b48a-1d8da84e37a4 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:33.920836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd96fae-d036-43d9-b7a3-f3e95624dee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:33.921726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96fae-d036-43d9-b7a3-f3e95624dee0 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:33.922222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96fae-d036-43d9-b7a3-f3e95624dee0 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:33.922707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96fae-d036-43d9-b7a3-f3e95624dee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:33.932678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd96fae-d036-43d9-b7a3-f3e95624dee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:33.934682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd96fae-d036-43d9-b7a3-f3e95624dee0 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:33.946452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 699/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:36.449542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694b36-8c45-4845-a5ff-56359b41e588 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:36.450000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694b36-8c45-4845-a5ff-56359b41e588 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:36.450236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb694b36-8c45-4845-a5ff-56359b41e588 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:36.451720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 687/1386] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:36.458239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09416605-359e-4dec-82d3-b3bce2d19431 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:36.458642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09416605-359e-4dec-82d3-b3bce2d19431 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:36.458807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09416605-359e-4dec-82d3-b3bce2d19431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:36.476963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-09416605-359e-4dec-82d3-b3bce2d19431 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:36.478143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-09416605-359e-4dec-82d3-b3bce2d19431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:36.488127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 700/1387] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:36.542592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694b36-8c45-4845-a5ff-56359b41e588 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:36.557483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb694b36-8c45-4845-a5ff-56359b41e588 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:36.563147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4800f0d-b2ba-48cb-b507-a0cc7ee5db4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:36.563779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4800f0d-b2ba-48cb-b507-a0cc7ee5db4a service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:36.564065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4800f0d-b2ba-48cb-b507-a0cc7ee5db4a service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:36.564342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4800f0d-b2ba-48cb-b507-a0cc7ee5db4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:36.572628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4800f0d-b2ba-48cb-b507-a0cc7ee5db4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:36.574416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4800f0d-b2ba-48cb-b507-a0cc7ee5db4a service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:36.584196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 688/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:39.407143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f3fb2-c98e-40c0-bb22-7eb2c29867ae None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:39.407690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f3fb2-c98e-40c0-bb22-7eb2c29867ae None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:39.407690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f3fb2-c98e-40c0-bb22-7eb2c29867ae None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:39.409064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 701/1389] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:39.414963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f0c49-dce9-4343-a40c-8949d25d5141 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:39.415379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f0c49-dce9-4343-a40c-8949d25d5141 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:39.415534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f0c49-dce9-4343-a40c-8949d25d5141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:39.433994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0c5f0c49-dce9-4343-a40c-8949d25d5141 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:39.435213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0c5f0c49-dce9-4343-a40c-8949d25d5141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:39.443105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 689/1390] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:39.487737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f3fb2-c98e-40c0-bb22-7eb2c29867ae None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:39.506341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f3fb2-c98e-40c0-bb22-7eb2c29867ae None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:39.513495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35635e60-61fa-4d4e-ba83-2ef00142beae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:39.514304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635e60-61fa-4d4e-ba83-2ef00142beae service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:39.514633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635e60-61fa-4d4e-ba83-2ef00142beae service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:39.515080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35635e60-61fa-4d4e-ba83-2ef00142beae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:39.525231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35635e60-61fa-4d4e-ba83-2ef00142beae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:39.526871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35635e60-61fa-4d4e-ba83-2ef00142beae service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:39.537036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 702/1391] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:41.829352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ea75-fdd6-4d4b-bb62-a412234e11e4 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:41.829956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ea75-fdd6-4d4b-bb62-a412234e11e4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:41.829956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77ea75-fdd6-4d4b-bb62-a412234e11e4 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:41.831320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 690/1392] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:41.836559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677c7e2-dcf6-4f4d-b991-8b1699e8a39d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:41.837015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677c7e2-dcf6-4f4d-b991-8b1699e8a39d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:41.837122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677c7e2-dcf6-4f4d-b991-8b1699e8a39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:41.854747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-5677c7e2-dcf6-4f4d-b991-8b1699e8a39d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:41.855974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5677c7e2-dcf6-4f4d-b991-8b1699e8a39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:41.863620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 703/1393] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:41.908045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ea75-fdd6-4d4b-bb62-a412234e11e4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:41.922288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77ea75-fdd6-4d4b-bb62-a412234e11e4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:41.927651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e9b1-0526-4596-8e8f-e1f4ea2771fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:41.928772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e9b1-0526-4596-8e8f-e1f4ea2771fd service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:41.929054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e9b1-0526-4596-8e8f-e1f4ea2771fd service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:41.929330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e9b1-0526-4596-8e8f-e1f4ea2771fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:41.937432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e9b1-0526-4596-8e8f-e1f4ea2771fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:41.939034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e9b1-0526-4596-8e8f-e1f4ea2771fd service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:41.949007 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 691/1394] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:43.093529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55445666-e3d2-4705-ad9b-afac9cd86bd7 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:43.094095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55445666-e3d2-4705-ad9b-afac9cd86bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:43.094095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55445666-e3d2-4705-ad9b-afac9cd86bd7 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:43.095576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 704/1395] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:43.101392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d0752-d9c5-4dfa-9bf0-3a951d0e38c9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:43.101804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d0752-d9c5-4dfa-9bf0-3a951d0e38c9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:43.101957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d0752-d9c5-4dfa-9bf0-3a951d0e38c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:43.113878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=54 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:10:43.121137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7c6d0752-d9c5-4dfa-9bf0-3a951d0e38c9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:43.122351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7c6d0752-d9c5-4dfa-9bf0-3a951d0e38c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:43.130299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 692/1396] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:43.185416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3814-202a-4719-ba50-34888a557e85 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:43.185712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3814-202a-4719-ba50-34888a557e85 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:43.185993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3814-202a-4719-ba50-34888a557e85 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:43.187393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 705/1397] 10.222.0.61 () {54 vars in 962 bytes} [Tue Jun 7 13:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:43.192441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f851-9cbe-4780-ab8f-6aa9d2dfcdd6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:43.192737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f851-9cbe-4780-ab8f-6aa9d2dfcdd6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:43.193018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f851-9cbe-4780-ab8f-6aa9d2dfcdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:43.242708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-adc8f851-9cbe-4780-ab8f-6aa9d2dfcdd6 None None] MFA Rules not processed for user `2fae9cab0b834583a01d0b57c78257af`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:43.243966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-adc8f851-9cbe-4780-ab8f-6aa9d2dfcdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:43.303976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 693/1398] 10.222.0.61 () {58 vars in 1059 bytes} [Tue Jun 7 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:43.311501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b3814-202a-4719-ba50-34888a557e85 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:43.321974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b3814-202a-4719-ba50-34888a557e85 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:43.325463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040e1a-fe64-4bda-8a5d-69f38e1fabe1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:43.326092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040e1a-fe64-4bda-8a5d-69f38e1fabe1 service glance] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:43.326379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040e1a-fe64-4bda-8a5d-69f38e1fabe1 service glance] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:43.326653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040e1a-fe64-4bda-8a5d-69f38e1fabe1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:43.335773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2040e1a-fe64-4bda-8a5d-69f38e1fabe1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:43.337208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2040e1a-fe64-4bda-8a5d-69f38e1fabe1 service glance] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:43.346696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 706/1399] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:43.870691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d8b27-bcc3-4b84-b21a-9778ecf3ab5b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:43.871009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d8b27-bcc3-4b84-b21a-9778ecf3ab5b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:43.871293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d8b27-bcc3-4b84-b21a-9778ecf3ab5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:43.923051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-225d8b27-bcc3-4b84-b21a-9778ecf3ab5b None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:43.924264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-225d8b27-bcc3-4b84-b21a-9778ecf3ab5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:43.988177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 694/1400] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:44.032112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556105a-8817-47ee-a541-cf9e25351d4c None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:44.032402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556105a-8817-47ee-a541-cf9e25351d4c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:44.032676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556105a-8817-47ee-a541-cf9e25351d4c None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:44.034039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 707/1401] 10.222.0.61 () {54 vars in 942 bytes} [Tue Jun 7 13:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:44.040097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc52ca-655b-453d-9243-28077a0c6277 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:44.040392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc52ca-655b-453d-9243-28077a0c6277 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:44.040677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc52ca-655b-453d-9243-28077a0c6277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:44.089899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-53dc52ca-655b-453d-9243-28077a0c6277 None None] MFA Rules not processed for user `67519cd46f644a03a35cf1151287752e`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:44.091113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-53dc52ca-655b-453d-9243-28077a0c6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:44.109965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:10:44.164941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 695/1402] 10.222.0.61 () {58 vars in 1039 bytes} [Tue Jun 7 13:10:44 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:44.174070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556105a-8817-47ee-a541-cf9e25351d4c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:44.187619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556105a-8817-47ee-a541-cf9e25351d4c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:44.192446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:44.193074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:44.193351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:44.194178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:44.202319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:44.203917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:44.206691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e319cb8-7700-4be0-97b5-160b14853810 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:44.207014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e319cb8-7700-4be0-97b5-160b14853810 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:44.207295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e319cb8-7700-4be0-97b5-160b14853810 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:44.209499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 696/1403] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:10:44.212630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 708/1404] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:10:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:10:44.214284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eae73-d9d1-4aee-9763-f795b105ea58 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:44.214714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eae73-d9d1-4aee-9763-f795b105ea58 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:44.214714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204eae73-d9d1-4aee-9763-f795b105ea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:44.233500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-204eae73-d9d1-4aee-9763-f795b105ea58 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:44.234704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-204eae73-d9d1-4aee-9763-f795b105ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:44.243297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 697/1405] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:44.288441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:44.300654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c74af-c98f-4dc6-bcdb-f13bfd9616bb service swift] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:44.314935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2a4da-7d35-4b4a-ab07-bc51259f414b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:44.315918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2a4da-7d35-4b4a-ab07-bc51259f414b service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:44.316539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2a4da-7d35-4b4a-ab07-bc51259f414b service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:44.317203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2a4da-7d35-4b4a-ab07-bc51259f414b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:44.328950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2a4da-7d35-4b4a-ab07-bc51259f414b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:44.330776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f2a4da-7d35-4b4a-ab07-bc51259f414b service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:44.355048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 709/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:46.666177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bae2f-da85-4661-b4c7-f0d3cfbedbd5 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:46.666702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bae2f-da85-4661-b4c7-f0d3cfbedbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:46.666702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bae2f-da85-4661-b4c7-f0d3cfbedbd5 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:46.668319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 698/1407] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:10:46.673586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898de53-c6cd-47b2-abc1-e9f49d4f19ef None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:46.674017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898de53-c6cd-47b2-abc1-e9f49d4f19ef None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:46.674210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898de53-c6cd-47b2-abc1-e9f49d4f19ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:46.693478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d898de53-c6cd-47b2-abc1-e9f49d4f19ef None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:10:46.695164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d898de53-c6cd-47b2-abc1-e9f49d4f19ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:10:46.703025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 710/1408] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:10:46.755257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bae2f-da85-4661-b4c7-f0d3cfbedbd5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:10:46.771138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bae2f-da85-4661-b4c7-f0d3cfbedbd5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:10:46.778234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7873f-3dcb-485a-86c5-06bedb6d4dba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:10:46.779870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7873f-3dcb-485a-86c5-06bedb6d4dba service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:10:46.780356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7873f-3dcb-485a-86c5-06bedb6d4dba service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:10:46.780692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7873f-3dcb-485a-86c5-06bedb6d4dba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:10:46.789881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7873f-3dcb-485a-86c5-06bedb6d4dba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:10:46.791579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7873f-3dcb-485a-86c5-06bedb6d4dba service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:10:46.801613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 699/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:10:56.690692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=84 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:10:56.796845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=31 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:11:07.650466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ccb9-1562-449b-952c-61d2a9a9a9dd None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:07.651068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ccb9-1562-449b-952c-61d2a9a9a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:07.651068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ccb9-1562-449b-952c-61d2a9a9a9dd None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:07.652472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 711/1410] 10.222.0.61 () {54 vars in 918 bytes} [Tue Jun 7 13:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:07.657539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df571-1807-49ee-8ddf-5348f4703117 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:07.657966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df571-1807-49ee-8ddf-5348f4703117 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:07.658102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df571-1807-49ee-8ddf-5348f4703117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:07.675930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-215df571-1807-49ee-8ddf-5348f4703117 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:07.677162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-215df571-1807-49ee-8ddf-5348f4703117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:07.685721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 700/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Jun 7 13:11:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:07.700229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882c15-9d4f-40af-bb8f-025a02d75857 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:07.700513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882c15-9d4f-40af-bb8f-025a02d75857 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:07.700787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882c15-9d4f-40af-bb8f-025a02d75857 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:07.702136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 712/1412] 10.222.0.61 () {54 vars in 918 bytes} [Tue Jun 7 13:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:07.707079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86b5b1-c7b7-4cf5-b471-57c2e6c8638f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:07.707377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86b5b1-c7b7-4cf5-b471-57c2e6c8638f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:07.707657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86b5b1-c7b7-4cf5-b471-57c2e6c8638f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:07.726327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cc86b5b1-c7b7-4cf5-b471-57c2e6c8638f None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:07.727607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cc86b5b1-c7b7-4cf5-b471-57c2e6c8638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:07.736734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 701/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Jun 7 13:11:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:10.454166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b49e42-b354-46f4-9eb9-87e0cd8618f6 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:10.454665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b49e42-b354-46f4-9eb9-87e0cd8618f6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:10.454764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b49e42-b354-46f4-9eb9-87e0cd8618f6 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:10.456385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:10.461743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07ae98-71e9-409f-8ac2-23a460701f00 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:10.462173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07ae98-71e9-409f-8ac2-23a460701f00 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:10.462325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07ae98-71e9-409f-8ac2-23a460701f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:10.480048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1a07ae98-71e9-409f-8ac2-23a460701f00 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:10.481224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1a07ae98-71e9-409f-8ac2-23a460701f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:10.488782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:10.530774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b49e42-b354-46f4-9eb9-87e0cd8618f6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:10.545895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b49e42-b354-46f4-9eb9-87e0cd8618f6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:10.551327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69e891-948b-47d0-bcf6-8c5ef7f27691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:10.551954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69e891-948b-47d0-bcf6-8c5ef7f27691 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:10.552236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69e891-948b-47d0-bcf6-8c5ef7f27691 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:10.552514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69e891-948b-47d0-bcf6-8c5ef7f27691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:10.561084 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69e891-948b-47d0-bcf6-8c5ef7f27691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:10.562647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69e891-948b-47d0-bcf6-8c5ef7f27691 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:10.571976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 714/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:11.543470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ad269-6cd5-4449-a4e9-67381b580010 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:11.544047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ad269-6cd5-4449-a4e9-67381b580010 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:11.544047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ad269-6cd5-4449-a4e9-67381b580010 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:11.545422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:11.550309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926edf4-27d9-4d4e-8177-ce07c43f82f5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:11.550724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926edf4-27d9-4d4e-8177-ce07c43f82f5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:11.550881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926edf4-27d9-4d4e-8177-ce07c43f82f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:11.568837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2926edf4-27d9-4d4e-8177-ce07c43f82f5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:11.570045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2926edf4-27d9-4d4e-8177-ce07c43f82f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:11.577636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:11.734327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ad269-6cd5-4449-a4e9-67381b580010 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:11.748600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ad269-6cd5-4449-a4e9-67381b580010 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:11.753796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb10ebd-a6ec-4fb1-915c-ae0dc959c548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:11.754417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb10ebd-a6ec-4fb1-915c-ae0dc959c548 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:11.754697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb10ebd-a6ec-4fb1-915c-ae0dc959c548 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:11.755017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb10ebd-a6ec-4fb1-915c-ae0dc959c548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:11.762972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb10ebd-a6ec-4fb1-915c-ae0dc959c548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:11.764529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb10ebd-a6ec-4fb1-915c-ae0dc959c548 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:11.774157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 704/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:12.692351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39892-9756-4ff1-99b5-a848cd462b13 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:12.692864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39892-9756-4ff1-99b5-a848cd462b13 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:12.692864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39892-9756-4ff1-99b5-a848cd462b13 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:12.694284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:12.699988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e0820-633d-4532-b41c-06ebe8abc611 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:12.700427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e0820-633d-4532-b41c-06ebe8abc611 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:12.700526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e0820-633d-4532-b41c-06ebe8abc611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:12.719163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-659e0820-633d-4532-b41c-06ebe8abc611 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:12.720359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-659e0820-633d-4532-b41c-06ebe8abc611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:12.728584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:12.877540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a39892-9756-4ff1-99b5-a848cd462b13 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:12.891721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a39892-9756-4ff1-99b5-a848cd462b13 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:12.897201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a693a3-4070-4a7c-b2c9-c738190a98b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:12.897827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a693a3-4070-4a7c-b2c9-c738190a98b9 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:12.898106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a693a3-4070-4a7c-b2c9-c738190a98b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:12.898377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a693a3-4070-4a7c-b2c9-c738190a98b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:12.906291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a693a3-4070-4a7c-b2c9-c738190a98b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:12.907883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a693a3-4070-4a7c-b2c9-c738190a98b9 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:12.917864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 717/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:13.811316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d3985-262c-471b-a345-d79f98facdbf None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:13.811872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d3985-262c-471b-a345-d79f98facdbf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:13.811966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d3985-262c-471b-a345-d79f98facdbf None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:13.813300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:13.818268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd88729-8af2-4dc0-8a74-d2437e15ba74 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:13.818694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd88729-8af2-4dc0-8a74-d2437e15ba74 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:13.818822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd88729-8af2-4dc0-8a74-d2437e15ba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:13.838805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-abd88729-8af2-4dc0-8a74-d2437e15ba74 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:13.840031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-abd88729-8af2-4dc0-8a74-d2437e15ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:13.847797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:13.995261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d3985-262c-471b-a345-d79f98facdbf None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:14.009912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d3985-262c-471b-a345-d79f98facdbf None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:14.015231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74b310-e3b8-4179-9c10-3e2874ea9a29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:14.015855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74b310-e3b8-4179-9c10-3e2874ea9a29 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:14.016135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74b310-e3b8-4179-9c10-3e2874ea9a29 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:14.016409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74b310-e3b8-4179-9c10-3e2874ea9a29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:14.024306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74b310-e3b8-4179-9c10-3e2874ea9a29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:14.025864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe74b310-e3b8-4179-9c10-3e2874ea9a29 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:14.034953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 707/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:14.929587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6fdcf-996e-402c-97c9-10f4f1bac74b None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:14.930157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6fdcf-996e-402c-97c9-10f4f1bac74b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:14.930157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6fdcf-996e-402c-97c9-10f4f1bac74b None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:14.931591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:14.936513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66301d-40ee-49a5-a8f2-e398901f5653 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:14.936944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66301d-40ee-49a5-a8f2-e398901f5653 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:14.937066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66301d-40ee-49a5-a8f2-e398901f5653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:14.955951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7d66301d-40ee-49a5-a8f2-e398901f5653 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:14.957176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7d66301d-40ee-49a5-a8f2-e398901f5653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:14.964864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:15.113567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6fdcf-996e-402c-97c9-10f4f1bac74b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:15.128467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6fdcf-996e-402c-97c9-10f4f1bac74b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:15.133934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89256bce-4668-48f1-9c19-8d90d1bfec45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:15.134562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89256bce-4668-48f1-9c19-8d90d1bfec45 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:15.134867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89256bce-4668-48f1-9c19-8d90d1bfec45 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:15.135147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89256bce-4668-48f1-9c19-8d90d1bfec45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:15.143283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89256bce-4668-48f1-9c19-8d90d1bfec45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:15.144873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89256bce-4668-48f1-9c19-8d90d1bfec45 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:15.154156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 720/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:16.002454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166a5e5-6694-45ea-a431-d4ec126a4195 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:16.003020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166a5e5-6694-45ea-a431-d4ec126a4195 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:16.003116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166a5e5-6694-45ea-a431-d4ec126a4195 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:16.004444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:16.009462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fddb7-be53-480f-aa3b-acae9e07ba62 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:16.009902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fddb7-be53-480f-aa3b-acae9e07ba62 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:16.010011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fddb7-be53-480f-aa3b-acae9e07ba62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:16.027648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-c85fddb7-be53-480f-aa3b-acae9e07ba62 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:16.028843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c85fddb7-be53-480f-aa3b-acae9e07ba62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:16.036639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:16.190677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166a5e5-6694-45ea-a431-d4ec126a4195 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:16.204638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166a5e5-6694-45ea-a431-d4ec126a4195 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:16.210795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de877106-a391-44b8-870c-fc2a904b8513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:16.211448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de877106-a391-44b8-870c-fc2a904b8513 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:16.211728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de877106-a391-44b8-870c-fc2a904b8513 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:16.211997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de877106-a391-44b8-870c-fc2a904b8513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:16.219936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de877106-a391-44b8-870c-fc2a904b8513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:16.221497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de877106-a391-44b8-870c-fc2a904b8513 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:16.231633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 710/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:17.074339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0db3c-8287-4736-8af5-046363174815 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:17.074935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0db3c-8287-4736-8af5-046363174815 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:17.074935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0db3c-8287-4736-8af5-046363174815 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:17.076303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:17.081615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b82ab-0755-41d1-abd3-baea9d2c9abf None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:17.082046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b82ab-0755-41d1-abd3-baea9d2c9abf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:17.082151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b82ab-0755-41d1-abd3-baea9d2c9abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:17.099626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c80b82ab-0755-41d1-abd3-baea9d2c9abf None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:17.100810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c80b82ab-0755-41d1-abd3-baea9d2c9abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:17.108642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:17.257662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0db3c-8287-4736-8af5-046363174815 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:17.271940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0db3c-8287-4736-8af5-046363174815 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:17.277367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5fe29f-6e27-4155-b6ea-a20a4faaabde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:17.277997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fe29f-6e27-4155-b6ea-a20a4faaabde service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:17.278285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fe29f-6e27-4155-b6ea-a20a4faaabde service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:17.278577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5fe29f-6e27-4155-b6ea-a20a4faaabde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:17.286666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fe29f-6e27-4155-b6ea-a20a4faaabde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:17.288282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5fe29f-6e27-4155-b6ea-a20a4faaabde service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:17.298105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 723/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:18.140968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed47b1-0e84-4f59-9b91-df01de8f0345 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:18.141380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed47b1-0e84-4f59-9b91-df01de8f0345 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:18.141524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed47b1-0e84-4f59-9b91-df01de8f0345 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:18.142930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:18.147953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e8e5b-2265-4546-ae5a-8bcc8962459e None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:18.148387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e8e5b-2265-4546-ae5a-8bcc8962459e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:18.148495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e8e5b-2265-4546-ae5a-8bcc8962459e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:18.166574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-af4e8e5b-2265-4546-ae5a-8bcc8962459e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:18.167786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-af4e8e5b-2265-4546-ae5a-8bcc8962459e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:18.175754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:18.325410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed47b1-0e84-4f59-9b91-df01de8f0345 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:18.340451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed47b1-0e84-4f59-9b91-df01de8f0345 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:18.345679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56250a-68a4-4d82-8531-e8f277c091ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:18.346302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56250a-68a4-4d82-8531-e8f277c091ea service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:18.346582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56250a-68a4-4d82-8531-e8f277c091ea service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:18.346878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56250a-68a4-4d82-8531-e8f277c091ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:18.355548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56250a-68a4-4d82-8531-e8f277c091ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:18.357134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56250a-68a4-4d82-8531-e8f277c091ea service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:18.366575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 713/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:19.236570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ee0d-47ee-49c0-aaca-b8729325af56 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:19.237134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ee0d-47ee-49c0-aaca-b8729325af56 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:19.237134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266ee0d-47ee-49c0-aaca-b8729325af56 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:19.238499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:19.243542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1675ad-5196-4cfc-8241-30ccf1b5d854 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:19.243988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1675ad-5196-4cfc-8241-30ccf1b5d854 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:19.244126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1675ad-5196-4cfc-8241-30ccf1b5d854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:19.261740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-da1675ad-5196-4cfc-8241-30ccf1b5d854 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:19.262978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-da1675ad-5196-4cfc-8241-30ccf1b5d854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:19.270682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:19.420119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ee0d-47ee-49c0-aaca-b8729325af56 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:19.434956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266ee0d-47ee-49c0-aaca-b8729325af56 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:19.440343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea2ac1-0d7a-453c-b2e8-bb511ed810d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:19.440986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea2ac1-0d7a-453c-b2e8-bb511ed810d7 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:19.441265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea2ac1-0d7a-453c-b2e8-bb511ed810d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:19.441537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea2ac1-0d7a-453c-b2e8-bb511ed810d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:19.450729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea2ac1-0d7a-453c-b2e8-bb511ed810d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:19.452354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea2ac1-0d7a-453c-b2e8-bb511ed810d7 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:19.462967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 726/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:21.551165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada560bd-d8ca-4157-b2c1-e6eb02978bac None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:21.551687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada560bd-d8ca-4157-b2c1-e6eb02978bac None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:21.551687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada560bd-d8ca-4157-b2c1-e6eb02978bac None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:21.553028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:21.558300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef2e61-4d98-4429-a73c-bc5b44d833c7 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:21.558725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef2e61-4d98-4429-a73c-bc5b44d833c7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:21.558854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef2e61-4d98-4429-a73c-bc5b44d833c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:21.575991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-06ef2e61-4d98-4429-a73c-bc5b44d833c7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:21.577164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-06ef2e61-4d98-4429-a73c-bc5b44d833c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:21.585182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:21.623107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada560bd-d8ca-4157-b2c1-e6eb02978bac None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:21.633543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada560bd-d8ca-4157-b2c1-e6eb02978bac None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:21.637250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5a899-615f-45f3-b01d-ad1ad267fb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:21.637918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a899-615f-45f3-b01d-ad1ad267fb08 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:21.638221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a899-615f-45f3-b01d-ad1ad267fb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:21.638480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5a899-615f-45f3-b01d-ad1ad267fb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:21.647543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5a899-615f-45f3-b01d-ad1ad267fb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:21.649725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5a899-615f-45f3-b01d-ad1ad267fb08 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:21.659029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 716/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:24.032336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf2be-8fe1-4971-a31b-3b256c3d95bf None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:24.032868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf2be-8fe1-4971-a31b-3b256c3d95bf None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:24.032868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf2be-8fe1-4971-a31b-3b256c3d95bf None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:24.034256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:24.040149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a0178-fc59-4035-a2de-0c674eb4d150 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:24.040589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a0178-fc59-4035-a2de-0c674eb4d150 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:24.040709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a0178-fc59-4035-a2de-0c674eb4d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:24.058520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ac4a0178-fc59-4035-a2de-0c674eb4d150 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:24.059749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ac4a0178-fc59-4035-a2de-0c674eb4d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:24.067622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:24.105751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf2be-8fe1-4971-a31b-3b256c3d95bf None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:24.116260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf2be-8fe1-4971-a31b-3b256c3d95bf None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:24.120389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162522f-1d8c-4bc3-8419-712c5ea9ba69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:24.121017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162522f-1d8c-4bc3-8419-712c5ea9ba69 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:24.121296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162522f-1d8c-4bc3-8419-712c5ea9ba69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:24.121568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162522f-1d8c-4bc3-8419-712c5ea9ba69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:24.130328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162522f-1d8c-4bc3-8419-712c5ea9ba69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:24.131779 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162522f-1d8c-4bc3-8419-712c5ea9ba69 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:24.141075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 729/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:26.730643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea7242-b53c-457d-9f73-a54c644a9444 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:26.731199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea7242-b53c-457d-9f73-a54c644a9444 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:26.731199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea7242-b53c-457d-9f73-a54c644a9444 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:26.732523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:26.737862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fbee-48b8-4775-b476-e811b7999efa None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:26.738296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fbee-48b8-4775-b476-e811b7999efa None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:26.738409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fbee-48b8-4775-b476-e811b7999efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:26.755699 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-90f0fbee-48b8-4775-b476-e811b7999efa None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:26.756889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-90f0fbee-48b8-4775-b476-e811b7999efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:26.764559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:26.803093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea7242-b53c-457d-9f73-a54c644a9444 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:26.813885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea7242-b53c-457d-9f73-a54c644a9444 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:26.817387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92107a-bb8c-42d4-9c3f-defdee679714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:26.818014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92107a-bb8c-42d4-9c3f-defdee679714 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:26.818293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92107a-bb8c-42d4-9c3f-defdee679714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:26.818566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92107a-bb8c-42d4-9c3f-defdee679714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:26.827314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92107a-bb8c-42d4-9c3f-defdee679714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:26.828721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92107a-bb8c-42d4-9c3f-defdee679714 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:26.838554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 719/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:29.209237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d2805-7784-4d00-a3eb-5882495fa169 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:29.209866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d2805-7784-4d00-a3eb-5882495fa169 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:29.209866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d2805-7784-4d00-a3eb-5882495fa169 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:29.211189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:29.216499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a1eba-226f-45b7-8f56-a77750d42b2c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:29.216955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a1eba-226f-45b7-8f56-a77750d42b2c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:29.217061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a1eba-226f-45b7-8f56-a77750d42b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:29.235282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3c4a1eba-226f-45b7-8f56-a77750d42b2c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:29.236720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3c4a1eba-226f-45b7-8f56-a77750d42b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:29.247331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:29.285500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d2805-7784-4d00-a3eb-5882495fa169 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:29.296661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d2805-7784-4d00-a3eb-5882495fa169 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:29.300146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234dd98-0ac2-4052-b5eb-ed18d2f7174f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:29.300769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234dd98-0ac2-4052-b5eb-ed18d2f7174f service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:29.301047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234dd98-0ac2-4052-b5eb-ed18d2f7174f service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:29.301319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234dd98-0ac2-4052-b5eb-ed18d2f7174f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:29.310749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234dd98-0ac2-4052-b5eb-ed18d2f7174f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:29.312181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234dd98-0ac2-4052-b5eb-ed18d2f7174f service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:29.321361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 732/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:31.633625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d858a5-5d2b-4e40-ba72-f2bb40a5df61 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:31.634196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d858a5-5d2b-4e40-ba72-f2bb40a5df61 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:31.634196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d858a5-5d2b-4e40-ba72-f2bb40a5df61 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:31.635599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:31.640866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134ca41-234f-47a0-af61-fcf5ad792cff None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:31.641275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134ca41-234f-47a0-af61-fcf5ad792cff None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:31.641391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134ca41-234f-47a0-af61-fcf5ad792cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:31.658609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e134ca41-234f-47a0-af61-fcf5ad792cff None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:31.659838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e134ca41-234f-47a0-af61-fcf5ad792cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:31.668045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:31.705735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d858a5-5d2b-4e40-ba72-f2bb40a5df61 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:31.716531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d858a5-5d2b-4e40-ba72-f2bb40a5df61 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:31.720004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb640762-a63d-4e77-b899-55b9e7fee04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:31.720626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb640762-a63d-4e77-b899-55b9e7fee04b service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:31.720915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb640762-a63d-4e77-b899-55b9e7fee04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:31.721214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb640762-a63d-4e77-b899-55b9e7fee04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:31.730463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb640762-a63d-4e77-b899-55b9e7fee04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:31.731911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb640762-a63d-4e77-b899-55b9e7fee04b service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:31.741248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 722/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:34.028739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bba0d-0ffc-402f-b95e-61ed73f00afb None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:34.029326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bba0d-0ffc-402f-b95e-61ed73f00afb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:34.029326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bba0d-0ffc-402f-b95e-61ed73f00afb None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:34.030649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:34.035910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45bac8-5deb-4844-9db0-b13967d1bda8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:34.036311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45bac8-5deb-4844-9db0-b13967d1bda8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:34.036465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45bac8-5deb-4844-9db0-b13967d1bda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:34.053689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0b45bac8-5deb-4844-9db0-b13967d1bda8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:34.055435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0b45bac8-5deb-4844-9db0-b13967d1bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:34.063080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:34.100562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bba0d-0ffc-402f-b95e-61ed73f00afb None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:34.111037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bba0d-0ffc-402f-b95e-61ed73f00afb None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:34.114491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f234f12-f43a-4bc3-9256-7c91afd9e185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:34.115145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234f12-f43a-4bc3-9256-7c91afd9e185 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:34.115428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234f12-f43a-4bc3-9256-7c91afd9e185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:34.115702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234f12-f43a-4bc3-9256-7c91afd9e185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:34.125017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f234f12-f43a-4bc3-9256-7c91afd9e185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:34.126447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f234f12-f43a-4bc3-9256-7c91afd9e185 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:34.136398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 735/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:36.432251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6baab-771f-4792-bc6d-f56301978494 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:36.432251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6baab-771f-4792-bc6d-f56301978494 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:36.432760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6baab-771f-4792-bc6d-f56301978494 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:36.433658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:36.439102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d397-bbab-42a1-877f-2aafe4fff6f9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:36.439587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d397-bbab-42a1-877f-2aafe4fff6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:36.439693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95d397-bbab-42a1-877f-2aafe4fff6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:36.458271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2c95d397-bbab-42a1-877f-2aafe4fff6f9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:36.459488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2c95d397-bbab-42a1-877f-2aafe4fff6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:36.467148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:36.504763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6baab-771f-4792-bc6d-f56301978494 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:36.506455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cee6baab-771f-4792-bc6d-f56301978494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:36.520050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6baab-771f-4792-bc6d-f56301978494 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:36.523610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bf30d-103e-4a02-a3a2-d79457b3edb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:36.524274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf30d-103e-4a02-a3a2-d79457b3edb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:36.524565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf30d-103e-4a02-a3a2-d79457b3edb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:36.524843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf30d-103e-4a02-a3a2-d79457b3edb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:36.533692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bf30d-103e-4a02-a3a2-d79457b3edb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:36.535148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bf30d-103e-4a02-a3a2-d79457b3edb5 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:36.544465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 725/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:38.829486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7784-da04-468b-97ce-13c2a8e40ed5 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:38.830021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7784-da04-468b-97ce-13c2a8e40ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:38.830021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7784-da04-468b-97ce-13c2a8e40ed5 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:38.831680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:38.837375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb8daf-6578-47c2-b368-7c9ecf1cd51c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:38.837809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb8daf-6578-47c2-b368-7c9ecf1cd51c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:38.837969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb8daf-6578-47c2-b368-7c9ecf1cd51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:38.844021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=79 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:11:38.857033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6bcb8daf-6578-47c2-b368-7c9ecf1cd51c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:38.858234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6bcb8daf-6578-47c2-b368-7c9ecf1cd51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:38.865936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:38.903013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7784-da04-468b-97ce-13c2a8e40ed5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:38.908384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=73 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:11:38.915501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab7784-da04-468b-97ce-13c2a8e40ed5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:38.918968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900cb39-1889-4352-8eee-5b86138c60ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:38.919598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900cb39-1889-4352-8eee-5b86138c60ad service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:38.919879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900cb39-1889-4352-8eee-5b86138c60ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:38.920152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900cb39-1889-4352-8eee-5b86138c60ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:38.929064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900cb39-1889-4352-8eee-5b86138c60ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:38.930489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900cb39-1889-4352-8eee-5b86138c60ad service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:38.939882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 738/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:41.229442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140162b-7d50-482b-be3f-2a0cb47e5142 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:41.229991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140162b-7d50-482b-be3f-2a0cb47e5142 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:41.230091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140162b-7d50-482b-be3f-2a0cb47e5142 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:41.231437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:41.238332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281cb15-da07-4866-8376-9f2c6c39fa1b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:41.238953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281cb15-da07-4866-8376-9f2c6c39fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:41.239106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281cb15-da07-4866-8376-9f2c6c39fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:41.257083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2281cb15-da07-4866-8376-9f2c6c39fa1b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:41.258964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2281cb15-da07-4866-8376-9f2c6c39fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:41.266648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:41.303009 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140162b-7d50-482b-be3f-2a0cb47e5142 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:41.314077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140162b-7d50-482b-be3f-2a0cb47e5142 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:41.317519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a4ff6-fde9-4d31-b64d-db42f902b279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:41.318141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4ff6-fde9-4d31-b64d-db42f902b279 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:41.318422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4ff6-fde9-4d31-b64d-db42f902b279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:41.318701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a4ff6-fde9-4d31-b64d-db42f902b279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:41.328216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4ff6-fde9-4d31-b64d-db42f902b279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:41.329628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a4ff6-fde9-4d31-b64d-db42f902b279 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:41.339112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 728/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:43.590819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd9c5e-64f9-447e-a81f-61647bc46bba None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:43.591406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd9c5e-64f9-447e-a81f-61647bc46bba None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:43.591406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd9c5e-64f9-447e-a81f-61647bc46bba None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:43.592846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:11:43.598018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e209d5-f5eb-46f1-aced-3de538b92ec6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:43.598458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e209d5-f5eb-46f1-aced-3de538b92ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:43.598581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e209d5-f5eb-46f1-aced-3de538b92ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:43.616518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-05e209d5-f5eb-46f1-aced-3de538b92ec6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:11:43.617733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-05e209d5-f5eb-46f1-aced-3de538b92ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:11:43.625479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:11:43.661343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd9c5e-64f9-447e-a81f-61647bc46bba None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:11:43.672469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd9c5e-64f9-447e-a81f-61647bc46bba None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:11:43.676193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7d73a-4c4e-4095-8c5e-ca04dcab9290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:11:43.676814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d73a-4c4e-4095-8c5e-ca04dcab9290 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:11:43.677093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d73a-4c4e-4095-8c5e-ca04dcab9290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:11:43.677367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d73a-4c4e-4095-8c5e-ca04dcab9290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:11:43.686022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7d73a-4c4e-4095-8c5e-ca04dcab9290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:11:43.687411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7d73a-4c4e-4095-8c5e-ca04dcab9290 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:11:43.696509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 741/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:11:53.611383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=11 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:11:53.692622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=14 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:12:15.395604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ad51f-231f-4414-abd5-2d1f092fa558 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:15.396222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ad51f-231f-4414-abd5-2d1f092fa558 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:15.396222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ad51f-231f-4414-abd5-2d1f092fa558 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:15.397539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 730/1471] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:15.403634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cfd18-3b82-469e-adc6-625c8bed6b3f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:15.404079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cfd18-3b82-469e-adc6-625c8bed6b3f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:15.404214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cfd18-3b82-469e-adc6-625c8bed6b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:15.421580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a49cfd18-3b82-469e-adc6-625c8bed6b3f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:15.422777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a49cfd18-3b82-469e-adc6-625c8bed6b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:15.430594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 742/1472] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:15.477387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018b7cc-0b31-4932-8ffe-ff0d15c8c144 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:15.477677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018b7cc-0b31-4932-8ffe-ff0d15c8c144 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:15.477983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018b7cc-0b31-4932-8ffe-ff0d15c8c144 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:15.479416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 731/1473] 10.222.0.61 () {54 vars in 962 bytes} [Tue Jun 7 13:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:15.484153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609931a-82f3-4bbe-94f0-8ea8a80ec080 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:15.484442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609931a-82f3-4bbe-94f0-8ea8a80ec080 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:15.484716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609931a-82f3-4bbe-94f0-8ea8a80ec080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:15.502052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a609931a-82f3-4bbe-94f0-8ea8a80ec080 None None] MFA Rules not processed for user `2fae9cab0b834583a01d0b57c78257af`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:15.503254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a609931a-82f3-4bbe-94f0-8ea8a80ec080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:15.510046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 743/1474] 10.222.0.61 () {58 vars in 1059 bytes} [Tue Jun 7 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:15.516912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018b7cc-0b31-4932-8ffe-ff0d15c8c144 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:15.527953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b018b7cc-0b31-4932-8ffe-ff0d15c8c144 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:15.531626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b57eed-1b07-4435-8fe0-3b640bfe220e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:15.532242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b57eed-1b07-4435-8fe0-3b640bfe220e service glance] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:15.532523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b57eed-1b07-4435-8fe0-3b640bfe220e service glance] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:15.532795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b57eed-1b07-4435-8fe0-3b640bfe220e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:15.542270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b57eed-1b07-4435-8fe0-3b640bfe220e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:15.543703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b57eed-1b07-4435-8fe0-3b640bfe220e service glance] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:15.552790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 732/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:12:17.860825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7207-1a93-49df-ac37-f45e476d3e28 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:17.861390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7207-1a93-49df-ac37-f45e476d3e28 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:17.861390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c7207-1a93-49df-ac37-f45e476d3e28 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:17.862803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 744/1476] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:12:17.868556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d3104-2844-49f6-b6d8-e57f692d8b9d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:17.868946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d3104-2844-49f6-b6d8-e57f692d8b9d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:17.869114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d3104-2844-49f6-b6d8-e57f692d8b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:17.886541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8d6d3104-2844-49f6-b6d8-e57f692d8b9d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:17.887765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8d6d3104-2844-49f6-b6d8-e57f692d8b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:17.895551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 733/1477] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:17.919023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c7207-1a93-49df-ac37-f45e476d3e28 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:17.931892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c7207-1a93-49df-ac37-f45e476d3e28 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:17.936444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:17.937085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:17.937366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:17.937639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:17.938367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:17.940125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:12:17.940125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:12:17.940125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:12:17.941241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fd772-6742-41ec-9c86-52fe8b2834eb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:17.960195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 745/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Tue Jun 7 13:12:17 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:12:20.190506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3886a91-7070-4f80-ac5f-c42f4b7aef24 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:20.190985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3886a91-7070-4f80-ac5f-c42f4b7aef24 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:20.191101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3886a91-7070-4f80-ac5f-c42f4b7aef24 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:20.192517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 734/1479] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:20.197685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2701f7-95e9-4f54-a6b4-594e435a97a3 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:20.198091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2701f7-95e9-4f54-a6b4-594e435a97a3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:20.198242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2701f7-95e9-4f54-a6b4-594e435a97a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:20.216605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3f2701f7-95e9-4f54-a6b4-594e435a97a3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:20.217810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3f2701f7-95e9-4f54-a6b4-594e435a97a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:20.225728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 746/1480] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:20.269286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3886a91-7070-4f80-ac5f-c42f4b7aef24 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:20.284092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3886a91-7070-4f80-ac5f-c42f4b7aef24 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:20.290121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fe013-7da2-43d5-895d-a4791d90ff40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:20.290752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fe013-7da2-43d5-895d-a4791d90ff40 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:20.291069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fe013-7da2-43d5-895d-a4791d90ff40 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:20.291381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fe013-7da2-43d5-895d-a4791d90ff40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:20.299471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fe013-7da2-43d5-895d-a4791d90ff40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:20.301044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fe013-7da2-43d5-895d-a4791d90ff40 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:20.310336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 735/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:12:22.736092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5209aa-9488-4df0-91c7-34661c081778 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:22.736619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5209aa-9488-4df0-91c7-34661c081778 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:22.736716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5209aa-9488-4df0-91c7-34661c081778 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:22.738072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 747/1482] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:22.743259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c708fc-ad6b-49f5-a5e6-ac40d14ed217 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:22.743638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c708fc-ad6b-49f5-a5e6-ac40d14ed217 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:22.743830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c708fc-ad6b-49f5-a5e6-ac40d14ed217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:22.761136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f4c708fc-ad6b-49f5-a5e6-ac40d14ed217 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:22.762329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f4c708fc-ad6b-49f5-a5e6-ac40d14ed217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:22.770051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 736/1483] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:22.806052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5209aa-9488-4df0-91c7-34661c081778 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:22.816708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5209aa-9488-4df0-91c7-34661c081778 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:22.820707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc6156-1661-4fc0-a3a4-01463aceb238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:22.821328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc6156-1661-4fc0-a3a4-01463aceb238 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:22.821609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc6156-1661-4fc0-a3a4-01463aceb238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:22.821883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc6156-1661-4fc0-a3a4-01463aceb238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:22.830661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc6156-1661-4fc0-a3a4-01463aceb238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:22.832104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc6156-1661-4fc0-a3a4-01463aceb238 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:22.842234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 748/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:12:25.108699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e970d40-fa0e-4753-9d80-da09f86d1a30 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:25.109277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e970d40-fa0e-4753-9d80-da09f86d1a30 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:25.109277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e970d40-fa0e-4753-9d80-da09f86d1a30 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:25.110644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 737/1485] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:25.116052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bc41f-457e-4170-87df-e5d2dc01fb50 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:25.116498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bc41f-457e-4170-87df-e5d2dc01fb50 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:25.116614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bc41f-457e-4170-87df-e5d2dc01fb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:25.135599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-fc8bc41f-457e-4170-87df-e5d2dc01fb50 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:25.136793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fc8bc41f-457e-4170-87df-e5d2dc01fb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:25.144526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 749/1486] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:25.181810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e970d40-fa0e-4753-9d80-da09f86d1a30 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:25.192616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e970d40-fa0e-4753-9d80-da09f86d1a30 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:25.196188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb10f8-6f7f-4857-82d5-72786b47f15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:25.196816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb10f8-6f7f-4857-82d5-72786b47f15b service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:25.197103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb10f8-6f7f-4857-82d5-72786b47f15b service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:25.197382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb10f8-6f7f-4857-82d5-72786b47f15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:25.206264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb10f8-6f7f-4857-82d5-72786b47f15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:25.207991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb10f8-6f7f-4857-82d5-72786b47f15b service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:25.217300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 738/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:12:27.592096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404def-efd5-4c0e-b38b-9c2c73912905 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:27.592676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404def-efd5-4c0e-b38b-9c2c73912905 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:27.592676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404def-efd5-4c0e-b38b-9c2c73912905 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:27.594098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 750/1488] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:27.599537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6c16a-6e60-43d8-b502-8ea46a587d94 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:27.599933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6c16a-6e60-43d8-b502-8ea46a587d94 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:27.600099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6c16a-6e60-43d8-b502-8ea46a587d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:27.618800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3fb6c16a-6e60-43d8-b502-8ea46a587d94 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:27.620023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3fb6c16a-6e60-43d8-b502-8ea46a587d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:27.627700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 739/1489] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:27.667051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404def-efd5-4c0e-b38b-9c2c73912905 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:27.677390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404def-efd5-4c0e-b38b-9c2c73912905 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:27.681474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b607652-c2e7-41b9-b1db-3158631b1a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:27.682145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b607652-c2e7-41b9-b1db-3158631b1a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:27.682370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b607652-c2e7-41b9-b1db-3158631b1a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:27.682638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b607652-c2e7-41b9-b1db-3158631b1a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:27.691396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b607652-c2e7-41b9-b1db-3158631b1a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:27.692808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b607652-c2e7-41b9-b1db-3158631b1a30 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:27.701981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 751/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:12:30.188125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3990dcc-cb7e-4698-ae3d-16d09b76c062 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:30.188582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3990dcc-cb7e-4698-ae3d-16d09b76c062 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:30.188692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3990dcc-cb7e-4698-ae3d-16d09b76c062 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:30.190120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 740/1491] 10.222.0.61 () {54 vars in 926 bytes} [Tue Jun 7 13:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:12:30.195854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8fb3-043d-4792-abdf-9f6be7662df9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:30.196271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8fb3-043d-4792-abdf-9f6be7662df9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:30.196410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d8fb3-043d-4792-abdf-9f6be7662df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:30.214096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-949d8fb3-043d-4792-abdf-9f6be7662df9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:12:30.215330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-949d8fb3-043d-4792-abdf-9f6be7662df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:12:30.223094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 752/1492] 10.222.0.61 () {58 vars in 1023 bytes} [Tue Jun 7 13:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:12:30.261768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3990dcc-cb7e-4698-ae3d-16d09b76c062 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:12:30.272274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3990dcc-cb7e-4698-ae3d-16d09b76c062 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:12:30.275804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71090cef-6c3c-4a49-8dc9-8c9af7ef9669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:12:30.276427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090cef-6c3c-4a49-8dc9-8c9af7ef9669 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:12:30.276710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090cef-6c3c-4a49-8dc9-8c9af7ef9669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:12:30.276988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71090cef-6c3c-4a49-8dc9-8c9af7ef9669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:12:30.285756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71090cef-6c3c-4a49-8dc9-8c9af7ef9669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:12:30.287204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71090cef-6c3c-4a49-8dc9-8c9af7ef9669 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:12:30.298244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 741/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:12:40.209010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=33 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:12:40.293649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=33 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:14:14.876352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:14.876950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:14.876950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:14.894903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:14:14.896087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:14.903780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 753/1494] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:14:14.911566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:14.912056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:14.912158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:14.930691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:14:14.932529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:14.940294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 742/1495] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:14:14.944758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:14.954268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce8bf4-95a4-437b-8c9e-35d26af0372f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:14.958679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:14.959357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:14.959639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:14.959909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:14.960623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:14:14.960923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:14.962181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:14.962181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:14:14.962181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:14:14.963074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:14.974204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 754/1496] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:14:14.982739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:14.992441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95e025-3714-4fc4-b435-1982c4babcff None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:14.997857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:14.998512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:14.998801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:14.999121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:14.999812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.001590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:15.001590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:14:15.001590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:14:15.002789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.036330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 743/1497] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:14:14 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:14:15.040572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.051146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf15cc5-9a0b-4b26-8524-ee0ff2037f7f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.106224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.106906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.107192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.107469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.108155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.109154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:15.109154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:14:15.109154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:14:15.110342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.130005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 755/1498] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:14:15 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:14:15.134176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.144128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fac10-b3bf-417a-92a5-c82ed42821a7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.148582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.149227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.149507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.149779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.150491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.151521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:15.151521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:14:15.151521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:14:15.152390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.164786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 744/1499] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:14:15.169108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.178973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40d144-271e-49d5-81e7-1593b3786114 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.184128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.184770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.185053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.185328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.186074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.187102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:15.187102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:14:15.187102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:14:15.187972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900cedd-8376-45da-a7af-40b7f696b7db admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.201104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 756/1500] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:14:15.205301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.216293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c1e71-c24f-45cb-ad53-b085da9ef590 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.220729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.221603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.221884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.222153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] PATH_INFO: `/v3/projects/eb4433bc85bc41a193806cabf49f8b53/users/35c33d1ceec8416b82c8b0416ece2123/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.236840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4433bc85bc41a193806cabf49f8b53, user_id=35c33d1ceec8416b82c8b0416ece2123, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.237865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:15.237865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:14:15.237865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:14:15.239827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00603e6-48a8-401b-99b6-80afb8ba8cdb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.254917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 745/1501] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:14:15 2022] PUT /identity/v3/projects/eb4433bc85bc41a193806cabf49f8b53/users/35c33d1ceec8416b82c8b0416ece2123/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:14:15.268150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6a84-1ac9-4974-b0a5-8dd89d73d128 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.268441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6a84-1ac9-4974-b0a5-8dd89d73d128 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.268712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da6a84-1ac9-4974-b0a5-8dd89d73d128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.295727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a0da6a84-1ac9-4974-b0a5-8dd89d73d128 None None] MFA Rules not processed for user `35c33d1ceec8416b82c8b0416ece2123`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:14:15.297011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a0da6a84-1ac9-4974-b0a5-8dd89d73d128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:15.358611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 757/1502] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:14:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:14:15.371809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205c36-ea52-401d-b7f2-26952b2dac16 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.372105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205c36-ea52-401d-b7f2-26952b2dac16 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.372412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205c36-ea52-401d-b7f2-26952b2dac16 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.373758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 746/1503] 10.222.0.61 () {54 vars in 963 bytes} [Tue Jun 7 13:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:14:15.378333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd560-b936-4f7a-b951-e582f65dabc0 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.378618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd560-b936-4f7a-b951-e582f65dabc0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.378924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd560-b936-4f7a-b951-e582f65dabc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.396292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-398fd560-b936-4f7a-b951-e582f65dabc0 None None] MFA Rules not processed for user `27c81d5738744687a915940977f0f647`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:14:15.397416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-398fd560-b936-4f7a-b951-e582f65dabc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:15.444463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:14:15.453144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 758/1504] 10.222.0.61 () {58 vars in 1060 bytes} [Tue Jun 7 13:14:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:14:15.460788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205c36-ea52-401d-b7f2-26952b2dac16 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.471976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205c36-ea52-401d-b7f2-26952b2dac16 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.475396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.476017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.476296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.476569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.485368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.486773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.495336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 747/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:14:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:14:15.537072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa40c65-24bc-4be0-b836-55f926769df8 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.537363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa40c65-24bc-4be0-b836-55f926769df8 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.537641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa40c65-24bc-4be0-b836-55f926769df8 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.538964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 759/1506] 10.222.0.61 () {52 vars in 829 bytes} [Tue Jun 7 13:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:14:15.543049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.602994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9073-b225-46ab-98c0-c261f014f371 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.608090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.608933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.609213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.609486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] PATH_INFO: `/v3/users/35c33d1ceec8416b82c8b0416ece2123` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.610651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c33d1ceec8416b82c8b0416ece2123)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.611687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:15.611687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:14:15.611687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:14:15.612882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.675930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c33d1ceec8416b82c8b0416ece2123'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:15.676598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c33d1ceec8416b82c8b0416ece2123'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:15.688592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c33d1ceec8416b82c8b0416ece2123'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:15.689254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c33d1ceec8416b82c8b0416ece2123'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:15.698907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c33d1ceec8416b82c8b0416ece2123'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:15.715640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:14:15.724661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 748/1507] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:14:15 2022] DELETE /identity/v3/users/35c33d1ceec8416b82c8b0416ece2123 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:14:15.728885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.729176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.729453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.746186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:14:15.747405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:15.803764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 760/1508] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:14:15.813182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:15.814566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:15.870966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5c4e5-535c-4565-8ff3-f283fa48645f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:15.874481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae413f5-f1c5-41aa-8126-5a1320fb42ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:15.875141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae413f5-f1c5-41aa-8126-5a1320fb42ea service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:15.875422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae413f5-f1c5-41aa-8126-5a1320fb42ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:15.875695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae413f5-f1c5-41aa-8126-5a1320fb42ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:15.884291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae413f5-f1c5-41aa-8126-5a1320fb42ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:15.885748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae413f5-f1c5-41aa-8126-5a1320fb42ea service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:15.894874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 749/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:14:16.235983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:14:16.237564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:14:16.249861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ba859-a6bd-47f1-a672-640706dcc8e1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:14:16.255285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:14:16.256053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:14:16.256334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:14:16.256599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] PATH_INFO: `/v3/projects/eb4433bc85bc41a193806cabf49f8b53` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:14:16.257789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4433bc85bc41a193806cabf49f8b53)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:14:16.258812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:14:16.258812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:14:16.258812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:14:16.260033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:14:16.321533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4433bc85bc41a193806cabf49f8b53', 'request_id': 'req-3051a802-973b-456e-af57-f3a198fbc14e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:16.331625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4433bc85bc41a193806cabf49f8b53', 'request_id': 'req-3051a802-973b-456e-af57-f3a198fbc14e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:16.338566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4433bc85bc41a193806cabf49f8b53', 'request_id': 'req-3051a802-973b-456e-af57-f3a198fbc14e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:16.357195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] The token cache is being invalidate because project eb4433bc85bc41a193806cabf49f8b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:14:16.357483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:16.358117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3051a802-973b-456e-af57-f3a198fbc14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:14:16.359785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 761/1510] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:14:16 2022] DELETE /identity/v3/projects/eb4433bc85bc41a193806cabf49f8b53 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:14:25.890814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:14:26.353516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:15:56.871311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4599b2-497c-4777-b029-a2f7afabc3ae None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:56.871994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4599b2-497c-4777-b029-a2f7afabc3ae None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:56.871994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4599b2-497c-4777-b029-a2f7afabc3ae None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:56.873905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:15:56.879239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c1d5-aebe-45b9-a778-99b89f89709f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:56.879780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c1d5-aebe-45b9-a778-99b89f89709f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:56.879780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283c1d5-aebe-45b9-a778-99b89f89709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:56.897427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3283c1d5-aebe-45b9-a778-99b89f89709f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:15:56.898622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3283c1d5-aebe-45b9-a778-99b89f89709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:15:56.957118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:15:57.126544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4599b2-497c-4777-b029-a2f7afabc3ae None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:15:57.128193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ac4599b2-497c-4777-b029-a2f7afabc3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:15:57.200270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4599b2-497c-4777-b029-a2f7afabc3ae None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:15:57.205648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a833962-3ecf-4693-8e09-eeb681f4532f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:15:57.206278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833962-3ecf-4693-8e09-eeb681f4532f service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:57.206565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833962-3ecf-4693-8e09-eeb681f4532f service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:57.206852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a833962-3ecf-4693-8e09-eeb681f4532f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:57.214781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a833962-3ecf-4693-8e09-eeb681f4532f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:15:57.217160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a833962-3ecf-4693-8e09-eeb681f4532f service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:15:57.226363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 751/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:15:58.091529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262bf3-ab6f-46ef-a8a3-3928305f8056 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:58.092189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262bf3-ab6f-46ef-a8a3-3928305f8056 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:58.092312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03262bf3-ab6f-46ef-a8a3-3928305f8056 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:58.093670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:15:58.098544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763348-0507-4580-9cd6-94bbe6c3e4c5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:58.098992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763348-0507-4580-9cd6-94bbe6c3e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:58.099215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763348-0507-4580-9cd6-94bbe6c3e4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:58.119111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4a763348-0507-4580-9cd6-94bbe6c3e4c5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:15:58.120299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4a763348-0507-4580-9cd6-94bbe6c3e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:15:58.128483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:15:58.285203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262bf3-ab6f-46ef-a8a3-3928305f8056 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:15:58.299874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03262bf3-ab6f-46ef-a8a3-3928305f8056 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:15:58.305229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54fdd6a-fb68-4784-b05b-54951ce2166b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:15:58.305862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fdd6a-fb68-4784-b05b-54951ce2166b service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:58.306164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fdd6a-fb68-4784-b05b-54951ce2166b service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:58.306403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fdd6a-fb68-4784-b05b-54951ce2166b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:58.315304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fdd6a-fb68-4784-b05b-54951ce2166b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:15:58.316913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54fdd6a-fb68-4784-b05b-54951ce2166b service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:15:58.326906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 764/1516] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:15:59.532838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d0e80-6386-47c9-83d6-7097e32b6393 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:59.533360 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d0e80-6386-47c9-83d6-7097e32b6393 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:59.533480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d0e80-6386-47c9-83d6-7097e32b6393 None None] PATH_INFO: `/v3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:59.534717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 753/1517] 10.222.0.18 () {54 vars in 925 bytes} [Tue Jun 7 13:15:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:15:59.541473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03df2a2-42d9-40c7-afe3-a9390a3ed295 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:15:59.541886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03df2a2-42d9-40c7-afe3-a9390a3ed295 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:15:59.542059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03df2a2-42d9-40c7-afe3-a9390a3ed295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:15:59.560300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b03df2a2-42d9-40c7-afe3-a9390a3ed295 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:15:59.561492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b03df2a2-42d9-40c7-afe3-a9390a3ed295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:15:59.621320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 765/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 7 13:15:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:03.856247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6d0a-7a4c-47ec-8a60-a2445f6c759f None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:03.857358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6d0a-7a4c-47ec-8a60-a2445f6c759f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:03.857358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f6d0a-7a4c-47ec-8a60-a2445f6c759f None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:03.858694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 754/1519] 10.222.0.61 () {54 vars in 960 bytes} [Tue Jun 7 13:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:03.864086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946899b-c44e-4fdc-b177-01f9a6a89b07 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:03.864518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946899b-c44e-4fdc-b177-01f9a6a89b07 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:03.864653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946899b-c44e-4fdc-b177-01f9a6a89b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:03.884146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3946899b-c44e-4fdc-b177-01f9a6a89b07 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:03.886673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3946899b-c44e-4fdc-b177-01f9a6a89b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:03.896864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 766/1520] 10.222.0.61 () {58 vars in 1057 bytes} [Tue Jun 7 13:16:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:03.904159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6d0a-7a4c-47ec-8a60-a2445f6c759f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:03.921186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f6d0a-7a4c-47ec-8a60-a2445f6c759f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:03.926369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f6612-3103-48a6-81d7-2258eb479b74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:03.927051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f6612-3103-48a6-81d7-2258eb479b74 service placement] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:03.927336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f6612-3103-48a6-81d7-2258eb479b74 service placement] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:03.927609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f6612-3103-48a6-81d7-2258eb479b74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:03.937090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f6612-3103-48a6-81d7-2258eb479b74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:03.938619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f6612-3103-48a6-81d7-2258eb479b74 service placement] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:03.948353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 755/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:09.592473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4679-4049-42a5-bdaf-91d0ffbdd278 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:09.593016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4679-4049-42a5-bdaf-91d0ffbdd278 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:09.593110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4679-4049-42a5-bdaf-91d0ffbdd278 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:09.594424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:09.599435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ab8eb-1a4d-4ed1-8941-4abd62ca2593 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:09.599851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ab8eb-1a4d-4ed1-8941-4abd62ca2593 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:09.599993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ab8eb-1a4d-4ed1-8941-4abd62ca2593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:09.618178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b54ab8eb-1a4d-4ed1-8941-4abd62ca2593 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:09.619404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b54ab8eb-1a4d-4ed1-8941-4abd62ca2593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:09.627237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:09.778003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b4679-4049-42a5-bdaf-91d0ffbdd278 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:09.793311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b4679-4049-42a5-bdaf-91d0ffbdd278 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:09.798736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faffd5-e0c7-4631-bc3f-ccb28eb2271c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:09.799393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faffd5-e0c7-4631-bc3f-ccb28eb2271c service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:09.799673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faffd5-e0c7-4631-bc3f-ccb28eb2271c service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:09.799944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faffd5-e0c7-4631-bc3f-ccb28eb2271c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:09.807918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faffd5-e0c7-4631-bc3f-ccb28eb2271c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:09.809538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faffd5-e0c7-4631-bc3f-ccb28eb2271c service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:09.818745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 768/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:10.688736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154291-6185-4c83-a24b-3f632a0c7ce8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:10.689270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154291-6185-4c83-a24b-3f632a0c7ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:10.689270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4154291-6185-4c83-a24b-3f632a0c7ce8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:10.690650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jun 7 13:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:10.695337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37082b45-1321-4501-8ea7-7b692ea6bcc9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:10.695740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37082b45-1321-4501-8ea7-7b692ea6bcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:10.695890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37082b45-1321-4501-8ea7-7b692ea6bcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:10.714036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-37082b45-1321-4501-8ea7-7b692ea6bcc9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:10.715302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-37082b45-1321-4501-8ea7-7b692ea6bcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:10.724021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jun 7 13:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:10.874024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154291-6185-4c83-a24b-3f632a0c7ce8 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:10.889015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4154291-6185-4c83-a24b-3f632a0c7ce8 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:10.896435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c134d6-a985-4604-a69a-7186b00c4868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:10.897071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c134d6-a985-4604-a69a-7186b00c4868 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:10.897357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c134d6-a985-4604-a69a-7186b00c4868 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:10.897632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c134d6-a985-4604-a69a-7186b00c4868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:10.906507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c134d6-a985-4604-a69a-7186b00c4868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:10.908131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c134d6-a985-4604-a69a-7186b00c4868 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:10.917578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 758/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:14.913756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b314e-496a-4ff6-b813-165addc6fa1e None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:14.914282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b314e-496a-4ff6-b813-165addc6fa1e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:14.914282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b314e-496a-4ff6-b813-165addc6fa1e None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:14.915708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:14.920843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708b276-9d97-457d-b147-2ab983c9230f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:14.921270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708b276-9d97-457d-b147-2ab983c9230f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:14.921436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708b276-9d97-457d-b147-2ab983c9230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:14.939029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c708b276-9d97-457d-b147-2ab983c9230f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:14.940226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c708b276-9d97-457d-b147-2ab983c9230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:14.948013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:14.984894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b314e-496a-4ff6-b813-165addc6fa1e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:14.986594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d25b314e-496a-4ff6-b813-165addc6fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:15.042158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b314e-496a-4ff6-b813-165addc6fa1e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:15.045752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83363369-f2b7-49fa-80ca-8127934d810b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:15.046385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83363369-f2b7-49fa-80ca-8127934d810b service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:15.046663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83363369-f2b7-49fa-80ca-8127934d810b service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:15.046966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83363369-f2b7-49fa-80ca-8127934d810b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:15.055621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83363369-f2b7-49fa-80ca-8127934d810b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:15.057037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83363369-f2b7-49fa-80ca-8127934d810b service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:15.061878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:16:15.075063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 771/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:17.322767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6deda-6070-4442-bd78-33afa14c99ad None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:17.323365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6deda-6070-4442-bd78-33afa14c99ad None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:17.323365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6deda-6070-4442-bd78-33afa14c99ad None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:17.324744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:17.330027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be9387-ed6c-459b-af31-5364b4745c89 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:17.330458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be9387-ed6c-459b-af31-5364b4745c89 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:17.330578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be9387-ed6c-459b-af31-5364b4745c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:17.348687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d5be9387-ed6c-459b-af31-5364b4745c89 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:17.349878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d5be9387-ed6c-459b-af31-5364b4745c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:17.358093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:17.393314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6deda-6070-4442-bd78-33afa14c99ad None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:17.403760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6deda-6070-4442-bd78-33afa14c99ad None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:17.407198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc10828-8fc8-4139-ae80-e09d5d625e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:17.407821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc10828-8fc8-4139-ae80-e09d5d625e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:17.408099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc10828-8fc8-4139-ae80-e09d5d625e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:17.408373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc10828-8fc8-4139-ae80-e09d5d625e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:17.417475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc10828-8fc8-4139-ae80-e09d5d625e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:17.419464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc10828-8fc8-4139-ae80-e09d5d625e2e service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:17.429165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 761/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:21.144035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af37b5e-f684-4194-824a-9049252f8d75 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:21.144631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af37b5e-f684-4194-824a-9049252f8d75 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:21.144631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af37b5e-f684-4194-824a-9049252f8d75 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:21.145943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:21.152230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada6eff-b22f-4d37-bf72-9473df1114da None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:21.152675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada6eff-b22f-4d37-bf72-9473df1114da None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:21.152789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada6eff-b22f-4d37-bf72-9473df1114da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:21.170735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4ada6eff-b22f-4d37-bf72-9473df1114da None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:21.171940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4ada6eff-b22f-4d37-bf72-9473df1114da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:21.179561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:21.454951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af37b5e-f684-4194-824a-9049252f8d75 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:21.456600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5af37b5e-f684-4194-824a-9049252f8d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:21.512993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af37b5e-f684-4194-824a-9049252f8d75 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:21.516531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91add329-ca3b-44bf-8f00-3fac5b21e481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:21.517165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91add329-ca3b-44bf-8f00-3fac5b21e481 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:21.517453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91add329-ca3b-44bf-8f00-3fac5b21e481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:21.517731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91add329-ca3b-44bf-8f00-3fac5b21e481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:21.526525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91add329-ca3b-44bf-8f00-3fac5b21e481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:21.527987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91add329-ca3b-44bf-8f00-3fac5b21e481 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:21.537782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 774/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:23.780089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e155c-7c54-4716-baf1-d73e524574da None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:23.780661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e155c-7c54-4716-baf1-d73e524574da None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:23.780661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e155c-7c54-4716-baf1-d73e524574da None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:23.782014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jun 7 13:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:16:23.787183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21071a-e3ee-4479-a16b-6efb634dd26f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:23.787636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21071a-e3ee-4479-a16b-6efb634dd26f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:23.787734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d21071a-e3ee-4479-a16b-6efb634dd26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:23.807209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6d21071a-e3ee-4479-a16b-6efb634dd26f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:16:23.808395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6d21071a-e3ee-4479-a16b-6efb634dd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:23.816476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jun 7 13:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:16:24.085289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e155c-7c54-4716-baf1-d73e524574da None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:16:24.086950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ad8e155c-7c54-4716-baf1-d73e524574da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:16:24.099778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e155c-7c54-4716-baf1-d73e524574da None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:16:24.103197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb32226-c949-48f4-b15e-a64507b6ba44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:16:24.103817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb32226-c949-48f4-b15e-a64507b6ba44 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:16:24.104098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb32226-c949-48f4-b15e-a64507b6ba44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:16:24.104369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb32226-c949-48f4-b15e-a64507b6ba44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:16:24.113476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb32226-c949-48f4-b15e-a64507b6ba44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:16:24.114900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb32226-c949-48f4-b15e-a64507b6ba44 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:16:24.124649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 764/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:16:33.800744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=32 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:16:34.120051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=70 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:55.582335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.583040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.583040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.612164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:55.613759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:55.625726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 776/1540] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:55.634733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.639382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.639677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.670617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:55.671796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:55.681810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 765/1541] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:55.686713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:55.697874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec2d34-494d-46d1-8586-bb0d6b7c34fd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:55.710542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:55.711651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.712222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.712704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.713678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:55.714202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:55.716066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:55.735551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 777/1542] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:55.745053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:55.765266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de916-3fff-485f-b86f-2d90386bf818 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:55.774287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f81fb1-005d-4d46-8cfb-d08382525b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:55.775116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f81fb1-005d-4d46-8cfb-d08382525b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.775399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f81fb1-005d-4d46-8cfb-d08382525b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.775672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f81fb1-005d-4d46-8cfb-d08382525b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.776559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f81fb1-005d-4d46-8cfb-d08382525b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:55.778537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f81fb1-005d-4d46-8cfb-d08382525b5f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:55.835482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 766/1543] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:55 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:55.840092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:55.855384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd41cd-3880-4970-b51d-68a8f9eda489 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:55.900285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838c6e3-ac24-47d6-9ea3-87fcdfbb64d0 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.900638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838c6e3-ac24-47d6-9ea3-87fcdfbb64d0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.900849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838c6e3-ac24-47d6-9ea3-87fcdfbb64d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.942954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4838c6e3-ac24-47d6-9ea3-87fcdfbb64d0 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:55.942954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4838c6e3-ac24-47d6-9ea3-87fcdfbb64d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:55.946803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:55.947898 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.948420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.948914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.949860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:55.952171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:55.968219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 767/1544] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:17:55.978004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:55.978411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:55.978634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:55.992789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 778/1545] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:55 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:55.998687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.011908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07787f34-dfc6-4622-bdea-c8aa3cbb5af4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.018732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.018732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.023778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.024742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.025252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.025734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.026684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.028714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.036576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 768/1546] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:17:56.044079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.045988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 779/1547] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:56.050880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.062381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5df92f-8bb4-4477-bc2f-b47675f69f13 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.063704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bf889-d2c1-416d-82d3-d4b5ba4695ff admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.075067 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9fe262-722d-42a3-82b6-4c3f5774995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.076237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9fe262-722d-42a3-82b6-4c3f5774995e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.076742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9fe262-722d-42a3-82b6-4c3f5774995e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.077238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9fe262-722d-42a3-82b6-4c3f5774995e admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.078193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9fe262-722d-42a3-82b6-4c3f5774995e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.080055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9fe262-722d-42a3-82b6-4c3f5774995e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.081470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.082141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.082425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.082736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.083502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.083808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.084821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:17:56.084821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:17:56.084821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:17:56.085708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.109118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 780/1548] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:17:56.110983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.111495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.112159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.121510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 769/1549] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.122357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.137863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.139585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.154005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874d79b-4c37-4f8b-ba86-c4e73dd7fcdb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.157290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 781/1550] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.158402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.169095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb3078-b9f0-456e-ba9a-6a3daf82dfe2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.178518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36732608-6438-4ee1-9c98-df05a96033ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.179562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732608-6438-4ee1-9c98-df05a96033ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.179847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732608-6438-4ee1-9c98-df05a96033ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.180124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732608-6438-4ee1-9c98-df05a96033ea admin admin] PATH_INFO: `/v3/projects/3408ff2ea39d4c428a122846f19a6809/users/07469976d61a438a92ed6d348621f032/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.181289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.182180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.182994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.183480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.184401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.185651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:17:56.185651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:17:56.185651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:17:56.187250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613dec1-8887-49dc-8bcc-e05c53fd212b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.229204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36732608-6438-4ee1-9c98-df05a96033ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3408ff2ea39d4c428a122846f19a6809, user_id=07469976d61a438a92ed6d348621f032, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.231970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36732608-6438-4ee1-9c98-df05a96033ea admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.247378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 782/1551] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:56.249644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.250205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.250723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.266072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 770/1552] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:56 2022] PUT /identity/v3/projects/3408ff2ea39d4c428a122846f19a6809/users/07469976d61a438a92ed6d348621f032/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:56.267561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5678507-edb0-4d61-8532-9b1e2d7d1628 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.267855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5678507-edb0-4d61-8532-9b1e2d7d1628 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.268622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5678507-edb0-4d61-8532-9b1e2d7d1628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.275180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.276749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.301309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e5678507-edb0-4d61-8532-9b1e2d7d1628 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.303446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e5678507-edb0-4d61-8532-9b1e2d7d1628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.355816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 783/1553] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.356726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.368036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cd772-802e-4b58-98cd-9c0c4bd42587 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.372851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895bb73-fec7-439c-8f08-8e26825131c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.373510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895bb73-fec7-439c-8f08-8e26825131c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.373791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895bb73-fec7-439c-8f08-8e26825131c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.374064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895bb73-fec7-439c-8f08-8e26825131c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.374754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895bb73-fec7-439c-8f08-8e26825131c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.376701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895bb73-fec7-439c-8f08-8e26825131c1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.377139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 771/1554] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.378607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c7ee6-e228-4c23-b5fa-44eb1aec2540 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.378970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c7ee6-e228-4c23-b5fa-44eb1aec2540 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.379256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c7ee6-e228-4c23-b5fa-44eb1aec2540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.398897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 784/1555] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:56.399105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bd8c7ee6-e228-4c23-b5fa-44eb1aec2540 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.399105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bd8c7ee6-e228-4c23-b5fa-44eb1aec2540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.399259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c602fe-f436-4585-9b66-5f0083613c8f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.399357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c602fe-f436-4585-9b66-5f0083613c8f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.399703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c602fe-f436-4585-9b66-5f0083613c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.405936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:56.411695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 772/1556] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.413805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8faae-65d8-439f-b776-259dddcfd4e5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.414295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8faae-65d8-439f-b776-259dddcfd4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.415358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8faae-65d8-439f-b776-259dddcfd4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.423913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d1c602fe-f436-4585-9b66-5f0083613c8f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.425216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d1c602fe-f436-4585-9b66-5f0083613c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.433933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 785/1557] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.435384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.435780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.436053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.444042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6be8faae-65d8-439f-b776-259dddcfd4e5 None None] MFA Rules not processed for user `07469976d61a438a92ed6d348621f032`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.445415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6be8faae-65d8-439f-b776-259dddcfd4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.455340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.456538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.464340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 786/1558] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.465162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.475206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa1e0f-f3e3-45e2-b2aa-ffd7239e5956 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.479717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.480311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.480590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.480862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.481580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.481880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.483533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5707-78b9-4e20-bff1-46f806fa68ee admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.495724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 787/1559] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.496998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.497294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.497566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.514319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 773/1560] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.515624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394edf-94e8-4261-9631-8a8af88b03b3 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.515917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394edf-94e8-4261-9631-8a8af88b03b3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.516141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.516251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394edf-94e8-4261-9631-8a8af88b03b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.517406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.523059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:56.526851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 788/1561] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.527583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.538346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b06730-5b75-4f2a-b6ec-f5d209123404 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.539606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-87394edf-94e8-4261-9631-8a8af88b03b3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.540789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-87394edf-94e8-4261-9631-8a8af88b03b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.543336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69a1da-4344-498f-889b-eee3d316aad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.544306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69a1da-4344-498f-889b-eee3d316aad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.544625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69a1da-4344-498f-889b-eee3d316aad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.544965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69a1da-4344-498f-889b-eee3d316aad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.545774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69a1da-4344-498f-889b-eee3d316aad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.547488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69a1da-4344-498f-889b-eee3d316aad8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.550779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 774/1562] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.552704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127459d2-3b3f-4420-8e81-7554a34b4f26 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.552990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127459d2-3b3f-4420-8e81-7554a34b4f26 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.553259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127459d2-3b3f-4420-8e81-7554a34b4f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.560872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 789/1563] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:56.560872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.561120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.561334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.574176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-127459d2-3b3f-4420-8e81-7554a34b4f26 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.575397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-127459d2-3b3f-4420-8e81-7554a34b4f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.579920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.581095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.583654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 775/1564] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.584948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd80f9-6595-4a74-82b8-56d8e6137baf None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.585240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd80f9-6595-4a74-82b8-56d8e6137baf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.585525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd80f9-6595-4a74-82b8-56d8e6137baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.589460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 790/1565] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.590219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.600226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80570b06-38cc-4fbf-9334-1508e75c68ee None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.604816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a150f3-69df-4452-a1a5-41e9964c0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.604982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a3fd80f9-6595-4a74-82b8-56d8e6137baf None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.605454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a150f3-69df-4452-a1a5-41e9964c0d61 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.605744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a150f3-69df-4452-a1a5-41e9964c0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.606016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a150f3-69df-4452-a1a5-41e9964c0d61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.606149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a3fd80f9-6595-4a74-82b8-56d8e6137baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.606707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a150f3-69df-4452-a1a5-41e9964c0d61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.608632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a150f3-69df-4452-a1a5-41e9964c0d61 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.613773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 776/1566] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.618081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.618081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.618081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.634574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.635790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.637856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 791/1567] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:56.639208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d1764-4429-42bf-950a-7a21ff6ed2bf None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.639507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d1764-4429-42bf-950a-7a21ff6ed2bf None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.639788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d1764-4429-42bf-950a-7a21ff6ed2bf None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.641066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 792/1568] 10.222.0.61 () {54 vars in 961 bytes} [Tue Jun 7 13:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:17:56.641751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d1764-4429-42bf-950a-7a21ff6ed2bf None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.696406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 777/1569] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.697594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.706557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea718e-49ce-4136-8ca1-0c0dd0c17f54 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.708432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d1764-4429-42bf-950a-7a21ff6ed2bf None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.712416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.713064 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.713199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.713347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.713619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.713716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.714002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.714280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.714374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.715046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.715357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.716233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.717113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.729627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 778/1570] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:56.730222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 793/1571] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.731109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.733432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.741349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88432e5-5675-4f14-a24f-d12e1a1d708d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.743174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445e6b7-fa7a-45a6-9b80-ca967e775645 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.745866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.746506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.746793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.747097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.747701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.747849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.748124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.748334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.748618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.748895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.749607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.749713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.749912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.751521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.761200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 794/1572] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.762048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.771330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 779/1573] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.772193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.772338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d3355-5b28-40ea-a340-f00a11009045 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.781868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.782511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.783028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.783304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.784023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.784351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.786499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff5495-311c-4026-b348-aa3452f5aa7d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.788333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cd049-2942-4f79-8df0-284cc472ac63 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.792980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.793619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.793906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.794185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.794893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.795896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:17:56.795896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:17:56.795896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:17:56.797092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.799401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 795/1574] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.799401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.799401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.799401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.818391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 780/1575] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:56.819308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.819987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:56.821181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:56.828877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d47ac-501a-4444-99a2-911d2bc9e6db admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.833888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.834518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.834800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.835094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.835803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:17:56.836102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.837657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.850454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 781/1576] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:17:56.851253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.860683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145c2ef-7a27-411c-b5ec-8f2607393db4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.865283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.866137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.866422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.866697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] PATH_INFO: `/v3/projects/56cf49983d984317b26f02dc0502e674/users/f9a3f501edab44559d30080706179006/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.879754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:56.880833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=56cf49983d984317b26f02dc0502e674, user_id=f9a3f501edab44559d30080706179006, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.883761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.897962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 782/1577] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:56 2022] PUT /identity/v3/projects/56cf49983d984317b26f02dc0502e674/users/f9a3f501edab44559d30080706179006/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:56.898377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 796/1578] 10.222.0.61 () {58 vars in 1058 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:56.898922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.899364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.909435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc07d66-a24b-4d95-bafa-14ceaa3b7828 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.961317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812594a-84a5-40a6-9b0b-994748ecd7ab admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:56.965875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.966033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:56.966520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.966628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:56.966803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.966934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:56.967110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.967214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:56.967789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.967887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:56.969956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.970072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:56.980497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:56.995913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 797/1579] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:56.996709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:56.997415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 783/1580] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:56.998275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.060159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f3a07-7a3c-47f7-a5fc-e62f4a657659 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.060796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b3dcf-a72c-4506-8976-351234fb1737 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.064794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.065449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.065580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.065736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.066013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.066212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.066412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.066686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.066792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.067961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.069163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.069841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.095674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 798/1581] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.096368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 784/1582] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:56 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.096504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.097152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.160616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d0454-9158-4b79-9b2a-b42c99893711 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.162641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0d237-edfe-41b0-b927-21b1a6398f2a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.165704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.166351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.166631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.167029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.167290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7aa08-9e25-46cd-90a3-d6140db3d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.167621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.167948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7aa08-9e25-46cd-90a3-d6140db3d09c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.168239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7aa08-9e25-46cd-90a3-d6140db3d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.168517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7aa08-9e25-46cd-90a3-d6140db3d09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.169312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7aa08-9e25-46cd-90a3-d6140db3d09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.169509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.170979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7aa08-9e25-46cd-90a3-d6140db3d09c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.172255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:57.186964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 799/1583] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.188318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.188650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.188947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.195285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 785/1584] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.199315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.214883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] MFA Rules not processed for user `f9a3f501edab44559d30080706179006`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:57.216112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:57.258772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:57.273629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b2a2d-c239-4fb7-8e54-e114335a387a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.279698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.279940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 800/1585] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:57.280760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.283295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.283594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.283932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.346340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.349002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.351139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ad07e-b3ee-4da4-a41b-d3a140c06e2f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.355706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.356372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.356659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.357071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.357766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.357897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 786/1586] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:57.358740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.360105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.369093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537ad14-e210-4919-b330-11d85a9881c2 service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.374941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.375607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.375886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.376231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.376875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.378753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.383452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 801/1587] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.384115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.396683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356301e-9fb6-4a2c-8f91-6993e1473c01 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.401796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.402464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.402750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.403077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.403776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.405694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.409351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 787/1588] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/users => generated 290 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.409351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.418374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213d9ad-7930-4052-abf6-486b6cf00a2f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.425302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.426030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.427276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.428219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.429220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.433028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 802/1589] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.433187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.433920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.444891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227363a8-a12c-4363-ad5f-d6e2b48e934d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.450783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa0e1a-e1b7-41d1-9996-9d49407f4ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.451544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0e1a-e1b7-41d1-9996-9d49407f4ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.451830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0e1a-e1b7-41d1-9996-9d49407f4ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.452107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa0e1a-e1b7-41d1-9996-9d49407f4ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.452859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa0e1a-e1b7-41d1-9996-9d49407f4ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.454693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa0e1a-e1b7-41d1-9996-9d49407f4ba3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.462056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 788/1590] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.462056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.468574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 803/1591] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.468574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b28eb-3fc9-41e1-a79d-09fc83192e79 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.468574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b28eb-3fc9-41e1-a79d-09fc83192e79 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.468574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b28eb-3fc9-41e1-a79d-09fc83192e79 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.469457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 804/1592] 10.222.0.61 () {54 vars in 961 bytes} [Tue Jun 7 13:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:17:57.470080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b28eb-3fc9-41e1-a79d-09fc83192e79 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.480274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee996be9-e925-4e57-8b10-62b9ee03f67c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.482507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b28eb-3fc9-41e1-a79d-09fc83192e79 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.485038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.485713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.485994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.486275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.487006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.487785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.488417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.488703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.489496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.489600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.490248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.491851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.504246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 805/1593] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.505490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.514501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 789/1594] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/users => generated 297 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:57.514501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.515741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e97b7-81b2-4a24-bf75-044d0682d66a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.520737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.521377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.521664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.521943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.522676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.524325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.524757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccffad3-37b6-42aa-a2de-5d20cc64f5dc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.530225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.530978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.531480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.531836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.532626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.535117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 806/1595] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.535253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.535337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.545037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e0b91-407b-41c7-a6e0-44a12cce4338 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.550886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 790/1596] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.550886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.553105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c149f-7716-436b-a864-df2985ede822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.553758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c149f-7716-436b-a864-df2985ede822 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.554043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c149f-7716-436b-a864-df2985ede822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.554320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c149f-7716-436b-a864-df2985ede822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.555073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c149f-7716-436b-a864-df2985ede822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.556674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917c149f-7716-436b-a864-df2985ede822 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.560426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed14df0-1ca9-42ec-a014-4c384f56241e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.565188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.566192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.566538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.566905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] PATH_INFO: `/v3/projects/2b82200b81a24815acd040d2e3b1e290/users/20a1930545d84798b7bd7053d05ee970/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.571591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 807/1597] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.571591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.571591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.571591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.587143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b82200b81a24815acd040d2e3b1e290, user_id=20a1930545d84798b7bd7053d05ee970, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.587831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:57.589047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:57.590408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.599391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 808/1598] 10.222.0.61 () {58 vars in 1058 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:57.600020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.610618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 791/1599] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:57 2022] PUT /identity/v3/projects/2b82200b81a24815acd040d2e3b1e290/users/20a1930545d84798b7bd7053d05ee970/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:57.610963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14e1bf-1b9b-4d0d-853c-0da402936817 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.611595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.615206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:57.688262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9ce51-ba31-45a6-8910-5ca608eefe3d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.691609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.692829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.693108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.693377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.694151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.694378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.694804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.695150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.695452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.695778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.696177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.697775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.705774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 809/1600] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.706631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.710744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 792/1601] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.710744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.717993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776a368-43f0-42e2-a9cf-954a260c5163 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.719444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0238ed-8f66-4d75-9b83-8a488304beae admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.722815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.723508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.723789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.724037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.724173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.724667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.724782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.724954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.725234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.725951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.726320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.727548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.728795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:57.741738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 810/1602] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.741738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.743000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 793/1603] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.743868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.749600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5755fe0-c807-45fe-994f-53e148479477 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.754281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab804d3-ef5b-4230-9d89-2f929904ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.755523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab804d3-ef5b-4230-9d89-2f929904ca06 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.755802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab804d3-ef5b-4230-9d89-2f929904ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.756072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab804d3-ef5b-4230-9d89-2f929904ca06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.756784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab804d3-ef5b-4230-9d89-2f929904ca06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.758357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab804d3-ef5b-4230-9d89-2f929904ca06 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.768054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 811/1604] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.769245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982957b8-4d01-415c-816b-5c79fdea5404 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.769744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982957b8-4d01-415c-816b-5c79fdea5404 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.769860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982957b8-4d01-415c-816b-5c79fdea5404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.794616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-982957b8-4d01-415c-816b-5c79fdea5404 None None] MFA Rules not processed for user `20a1930545d84798b7bd7053d05ee970`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:57.795942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-982957b8-4d01-415c-816b-5c79fdea5404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:57.814242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01da08-5345-4e13-b4ed-8bfbf2fa0848 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.820200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.820835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.821116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.821392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.860740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 812/1605] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:57.862183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404223f8-8919-4082-8d8d-c93a7101028f None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.862471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404223f8-8919-4082-8d8d-c93a7101028f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.862748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404223f8-8919-4082-8d8d-c93a7101028f None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.864894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 813/1606] 10.222.0.61 () {54 vars in 912 bytes} [Tue Jun 7 13:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:17:57.865642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404223f8-8919-4082-8d8d-c93a7101028f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.877860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404223f8-8919-4082-8d8d-c93a7101028f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.882580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.882916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.883751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.884031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.884212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.884322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] PATH_INFO: `/v3/projects/ec73e3d0e6f8476d81463ea6e11b62cd/users/49c60c4956f8479fbfe93f9f85a8d497/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.893460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 794/1607] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:57.894150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.899511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec73e3d0e6f8476d81463ea6e11b62cd, user_id=49c60c4956f8479fbfe93f9f85a8d497, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.900543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:17:57.900543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:17:57.900543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:17:57.902444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.904481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a97048-8435-497a-b50b-da7e70860434 service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.909688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.910347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.910647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.910978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.911728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:57.913716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:57.918091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 814/1608] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:57 2022] PUT /identity/v3/projects/ec73e3d0e6f8476d81463ea6e11b62cd/users/49c60c4956f8479fbfe93f9f85a8d497/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:57.918866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.926984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 795/1609] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:57.928843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:57.939639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e4ff1-9952-4524-bbae-2875a6b5d0bd admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.969535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:57.987434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ea21b-7f65-44ae-bdaa-6e9c74182c65 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:57.993145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.994097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.994515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:57.995043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] PATH_INFO: `/v3/projects/7b95f79de5724987ab6c9fddb86a84f0/users/106d1325381147b1ba157f2d54d0388c/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:57.998402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea744be-6e7b-4e22-b3f9-af75810150d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:57.999430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea744be-6e7b-4e22-b3f9-af75810150d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:57.999818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea744be-6e7b-4e22-b3f9-af75810150d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.000195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea744be-6e7b-4e22-b3f9-af75810150d6 admin admin] PATH_INFO: `/v3/projects/28f4c96b72d341609bfb30ec3d20faae/users/dbc72739a09a4d65a064df2d912226f7/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.012459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b95f79de5724987ab6c9fddb86a84f0, user_id=106d1325381147b1ba157f2d54d0388c, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.015559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea744be-6e7b-4e22-b3f9-af75810150d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f4c96b72d341609bfb30ec3d20faae, user_id=dbc72739a09a4d65a064df2d912226f7, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.015996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.018519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea744be-6e7b-4e22-b3f9-af75810150d6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.031996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 815/1610] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:57 2022] PUT /identity/v3/projects/7b95f79de5724987ab6c9fddb86a84f0/users/106d1325381147b1ba157f2d54d0388c/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:58.035767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.036363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 796/1611] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:57 2022] PUT /identity/v3/projects/28f4c96b72d341609bfb30ec3d20faae/users/dbc72739a09a4d65a064df2d912226f7/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:58.036856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd1ee7-cb60-4f11-97a7-9e68c8454913 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.037228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd1ee7-cb60-4f11-97a7-9e68c8454913 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.037603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd1ee7-cb60-4f11-97a7-9e68c8454913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.051562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:58.065071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a7bd1ee7-cb60-4f11-97a7-9e68c8454913 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.066275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a7bd1ee7-cb60-4f11-97a7-9e68c8454913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.109363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c13be-0102-464b-a3dd-f6d73efd8037 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.136191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 797/1612] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.137967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb46c-c6f3-4eae-81fe-b25168d52b3c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.137967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb46c-c6f3-4eae-81fe-b25168d52b3c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.138372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571eb46c-c6f3-4eae-81fe-b25168d52b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.157368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-571eb46c-c6f3-4eae-81fe-b25168d52b3c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.158655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-571eb46c-c6f3-4eae-81fe-b25168d52b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.169656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.170544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.170844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.171140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] PATH_INFO: `/v3/projects/5729496a93514450b7b7cda4ad736ba6/users/e56fdbebcd3741ae98874e51a8cc9104/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.185850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5729496a93514450b7b7cda4ad736ba6, user_id=e56fdbebcd3741ae98874e51a8cc9104, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.188545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.201096 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 816/1613] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:58 2022] PUT /identity/v3/projects/5729496a93514450b7b7cda4ad736ba6/users/e56fdbebcd3741ae98874e51a8cc9104/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:58.208492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.219878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 798/1614] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.221170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122be70-3c5a-46b6-9f48-0ce556665fce None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.221466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122be70-3c5a-46b6-9f48-0ce556665fce None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.221748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122be70-3c5a-46b6-9f48-0ce556665fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.241013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6122be70-3c5a-46b6-9f48-0ce556665fce None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.242214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6122be70-3c5a-46b6-9f48-0ce556665fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.275171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9b7c2-1d4c-428b-8ca0-3fe1fd128158 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.279794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080117d1-1c4a-4446-a409-e83c3cf95aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.280707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080117d1-1c4a-4446-a409-e83c3cf95aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.280999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080117d1-1c4a-4446-a409-e83c3cf95aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.281282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080117d1-1c4a-4446-a409-e83c3cf95aac admin admin] PATH_INFO: `/v3/projects/7c615cd1e80a42e0ac2aff126cbb1455/users/dcf5286e9ca14f56905e6e247e411896/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.283557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:58.297652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080117d1-1c4a-4446-a409-e83c3cf95aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c615cd1e80a42e0ac2aff126cbb1455, user_id=dcf5286e9ca14f56905e6e247e411896, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.300333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080117d1-1c4a-4446-a409-e83c3cf95aac admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.304114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 799/1615] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.305532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741aa9-a4ff-4fb7-979e-0b66b5355f24 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.305848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741aa9-a4ff-4fb7-979e-0b66b5355f24 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.306183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34741aa9-a4ff-4fb7-979e-0b66b5355f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.318702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 817/1616] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:58 2022] PUT /identity/v3/projects/7c615cd1e80a42e0ac2aff126cbb1455/users/dcf5286e9ca14f56905e6e247e411896/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:58.318702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e154c77-ebf8-4c4f-a36a-80df5a8fb9e8 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.318702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e154c77-ebf8-4c4f-a36a-80df5a8fb9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.318702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e154c77-ebf8-4c4f-a36a-80df5a8fb9e8 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.318702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 818/1617] 10.222.0.61 () {54 vars in 912 bytes} [Tue Jun 7 13:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:17:58.319390 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e154c77-ebf8-4c4f-a36a-80df5a8fb9e8 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.336535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-34741aa9-a4ff-4fb7-979e-0b66b5355f24 None None] MFA Rules not processed for user `dbc72739a09a4d65a064df2d912226f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.337749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-34741aa9-a4ff-4fb7-979e-0b66b5355f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.364433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:58.377927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e154c77-ebf8-4c4f-a36a-80df5a8fb9e8 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.382132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.382906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.383193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.383473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.405001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 800/1618] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.410256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.410582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.411042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.450355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] MFA Rules not processed for user `e56fdbebcd3741ae98874e51a8cc9104`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.451627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.463040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.464510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.475804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 819/1619] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:58.476614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.477911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.520325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 801/1620] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.521182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.522493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.540263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442ea79-5cea-4c77-bee8-54da79d83ffd service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.544425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee98258-6516-446e-a89f-74624edcb6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.545370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98258-6516-446e-a89f-74624edcb6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.545820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98258-6516-446e-a89f-74624edcb6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.546261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98258-6516-446e-a89f-74624edcb6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.587723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23aca96-ae18-4ca9-8a4c-6a61bf46c992 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.590689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.591427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.592033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.592158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:58.592257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.617685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee98258-6516-446e-a89f-74624edcb6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.619166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee98258-6516-446e-a89f-74624edcb6c8 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.629347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 820/1621] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:58.630803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cbe1b-6b1d-471b-9af7-48ffef00ba31 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.631188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cbe1b-6b1d-471b-9af7-48ffef00ba31 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.631388 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cbe1b-6b1d-471b-9af7-48ffef00ba31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.648187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f02cbe1b-6b1d-471b-9af7-48ffef00ba31 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.649669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f02cbe1b-6b1d-471b-9af7-48ffef00ba31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.660400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 821/1622] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.662263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.662554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.662853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.672461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.673930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.682660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:58.683748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:58.685719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 802/1623] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:58.686701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.693907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 822/1624] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:58.694609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.703335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe99e59-7e7d-43eb-b1cf-750da68d9c54 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.705861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4111d-ca85-4409-8e7c-518dbb069a93 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.709611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.710543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.710936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.711437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.711650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.711979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.712214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.712589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.712889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.713701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.714787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.716219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.721201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:58.745232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 823/1625] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:58.747922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.749231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 803/1626] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:17:58 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:58.750253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.826181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b2415-6039-4951-9842-5e026fe50fce admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.832834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.833670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.834076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.834213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560d952-e612-4736-9c1b-fe9d85571e87 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:58.834600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.908640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:58.910342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:58.919132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:58.919673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:58.920023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:58.920274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:58.922931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 804/1627] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:58.923808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:58.985837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:59.011583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb48606-4029-41db-bbe5-28f57a4c238d service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.017364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.018163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.018370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.018635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.033929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.040111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.052096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.054271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.060153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 824/1628] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:59.061170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.070860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 805/1629] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:17:59.071800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.075497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdee46b-b706-4661-8ed8-19608aef781b service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.081567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.082242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.082520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.082868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.083697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.086250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.100582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79141e19-9a5a-45e5-b60a-51c0f0233b76 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.115548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.116242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.116582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.116799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.117657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.120072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.124815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 825/1630] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:59 2022] POST /identity/v3/users => generated 292 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:59.132148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.152702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bb6fd-2a85-4562-b39f-ff5a64554ae5 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.159268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.159951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.160238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.160995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.161321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.165678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 806/1631] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:17:59 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:17:59.169053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.169289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.186879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 826/1632] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:59.187056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d9748-1b2a-4150-a57b-8bf55649074d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.191267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.209593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.210284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.210576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.210927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.211627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.212188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57dcd8-c3ad-405e-b2fb-28a49c13a56f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.213263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.218427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.219864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.220150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.220700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] PATH_INFO: `/v3/projects/2d09c41d6ea14efaa98f4a964ccc928b/users/2ec748790a894fa3b9751942c3c6011d/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.244210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d09c41d6ea14efaa98f4a964ccc928b, user_id=2ec748790a894fa3b9751942c3c6011d, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.248670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.250302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 807/1633] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:59.254466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.269603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039536f-8fa9-41c2-ab36-32f579264883 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.285614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:59.291629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 827/1634] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:59 2022] PUT /identity/v3/projects/2d09c41d6ea14efaa98f4a964ccc928b/users/2ec748790a894fa3b9751942c3c6011d/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:59.293620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.451399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.453291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.453649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.454248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] PATH_INFO: `/v3/projects/a6431fdebb5e4ba7b978d361a203a504/users/7357dfac388640d9a84ddda45dd18bff/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.495255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281944-ace6-4e02-825d-fa912bb57316 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.500402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6431fdebb5e4ba7b978d361a203a504, user_id=7357dfac388640d9a84ddda45dd18bff, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.508259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.509794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.510941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.511269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.512187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.513941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.516585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.553145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 808/1635] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:59 2022] PUT /identity/v3/projects/a6431fdebb5e4ba7b978d361a203a504/users/7357dfac388640d9a84ddda45dd18bff/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:17:59.554367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.558670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 828/1636] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:59.562302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.586234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366a623-a39c-4551-ac0b-410b7093e6fe admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.662117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684f498-b276-42e0-93f4-d40ba870bddc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.676899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.678020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.678519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.678971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.680437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.685378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.704848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e29e04-f1b9-4bd9-9188-0c3299fb63c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.705919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29e04-f1b9-4bd9-9188-0c3299fb63c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.706388 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29e04-f1b9-4bd9-9188-0c3299fb63c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.706794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e29e04-f1b9-4bd9-9188-0c3299fb63c0 admin admin] PATH_INFO: `/v3/projects/2d09c41d6ea14efaa98f4a964ccc928b/users/2ec748790a894fa3b9751942c3c6011d/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.711241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29e04-f1b9-4bd9-9188-0c3299fb63c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d09c41d6ea14efaa98f4a964ccc928b, user_id=2ec748790a894fa3b9751942c3c6011d, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.714671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e29e04-f1b9-4bd9-9188-0c3299fb63c0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.716199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 809/1637] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:17:59.717154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:17:59.740938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109b9ea-035b-4a36-bc4e-5aaca57a945b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:17:59.741455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 829/1638] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:59 2022] PUT /identity/v3/projects/2d09c41d6ea14efaa98f4a964ccc928b/users/2ec748790a894fa3b9751942c3c6011d/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:59.751936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.752226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.752549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.802647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:17:59.827598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] MFA Rules not processed for user `2ec748790a894fa3b9751942c3c6011d`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:59.830477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:59.841475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8306fde-f60a-4f93-a34d-10e06e8aa8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:17:59.842708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8306fde-f60a-4f93-a34d-10e06e8aa8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.843032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8306fde-f60a-4f93-a34d-10e06e8aa8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.843505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8306fde-f60a-4f93-a34d-10e06e8aa8a7 admin admin] PATH_INFO: `/v3/projects/a6431fdebb5e4ba7b978d361a203a504/users/7357dfac388640d9a84ddda45dd18bff/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.846181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8306fde-f60a-4f93-a34d-10e06e8aa8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6431fdebb5e4ba7b978d361a203a504, user_id=7357dfac388640d9a84ddda45dd18bff, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:17:59.849385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8306fde-f60a-4f93-a34d-10e06e8aa8a7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:17:59.868699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 810/1639] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:17:59 2022] PUT /identity/v3/projects/a6431fdebb5e4ba7b978d361a203a504/users/7357dfac388640d9a84ddda45dd18bff/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:17:59.874702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4dc9f-b7df-4cce-95b6-c4fb6183158d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:17:59.875045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4dc9f-b7df-4cce-95b6-c4fb6183158d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:17:59.875324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4dc9f-b7df-4cce-95b6-c4fb6183158d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:17:59.945397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bdb4dc9f-b7df-4cce-95b6-c4fb6183158d None None] MFA Rules not processed for user `7357dfac388640d9a84ddda45dd18bff`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:17:59.946858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bdb4dc9f-b7df-4cce-95b6-c4fb6183158d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:17:59.964459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 830/1640] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:17:59.970231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:00.057065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:00.168052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 811/1641] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:17:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:00.194328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc92c7-1023-456a-8245-0104a7bd182d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:00.281892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:00.282784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:00.283099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:00.283499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:00.403015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:00.404489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:00.439253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 831/1642] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:00.446267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556b261-1f8c-4d7e-a1f7-3c10f7470586 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:00.446556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556b261-1f8c-4d7e-a1f7-3c10f7470586 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:00.446869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556b261-1f8c-4d7e-a1f7-3c10f7470586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:00.504055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9556b261-1f8c-4d7e-a1f7-3c10f7470586 None None] MFA Rules not processed for user `106d1325381147b1ba157f2d54d0388c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:00.505789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9556b261-1f8c-4d7e-a1f7-3c10f7470586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:00.580057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:00.585652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:00.591928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91c134-4d85-49db-ac65-6777ee8cba8b service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:00.597867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:00.599154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:00.599847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:00.599847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:00.612535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:00.614465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:00.620410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 812/1643] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:00.631116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:00.633008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:00.633462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:00.644890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 832/1644] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:00.693766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] MFA Rules not processed for user `106d1325381147b1ba157f2d54d0388c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:00.697371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:00.714872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 813/1645] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:00.889272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:00.912760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:00.927124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05351321-e775-4e05-91e5-729fa20b9e70 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:00.932672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:00.935016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:00.935586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:00.935868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:00.999647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7f0de-74c1-414a-91b7-ef476b9935c6 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:01.008796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:01.009525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:01.009980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:01.010311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:01.031859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:01.035308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:01.036872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:01.040120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:01.050463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 833/1646] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:01.058132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 814/1647] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:01.091543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:01.102580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c7505-d68b-4ca7-a259-50cfa03c0488 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:01.118509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2ea82-bd0a-4b65-aa90-b04bf267f2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:01.119200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ea82-bd0a-4b65-aa90-b04bf267f2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:01.119484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ea82-bd0a-4b65-aa90-b04bf267f2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:01.119758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2ea82-bd0a-4b65-aa90-b04bf267f2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:01.235484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ea82-bd0a-4b65-aa90-b04bf267f2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:01.237124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2ea82-bd0a-4b65-aa90-b04bf267f2d6 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:01.253036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 834/1648] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:01.635422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:01.716636 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:01.771654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536548ab-2513-4dfc-b970-b70806238c55 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:01.785238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:01.789047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:01.789994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:01.790333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] PATH_INFO: `/v3/users/07469976d61a438a92ed6d348621f032` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:01.792013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07469976d61a438a92ed6d348621f032)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:01.794989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:01.848178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de2ec-5820-4a90-9395-349f656d2ff6 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:01.849302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de2ec-5820-4a90-9395-349f656d2ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:01.849744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de2ec-5820-4a90-9395-349f656d2ff6 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:01.851332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 835/1649] 10.222.0.61 () {54 vars in 912 bytes} [Tue Jun 7 13:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:18:01.857823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:01.858346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:01.858804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:01.868384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07469976d61a438a92ed6d348621f032'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:01.869306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07469976d61a438a92ed6d348621f032'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:01.890524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07469976d61a438a92ed6d348621f032'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:01.893133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07469976d61a438a92ed6d348621f032'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:01.921029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:01.921588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b0200d4c-55fe-4f06-a25e-94e3a9112152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07469976d61a438a92ed6d348621f032'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:01.923343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:01.967102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 815/1650] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:01 2022] DELETE /identity/v3/users/07469976d61a438a92ed6d348621f032 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:01.972585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:01.972944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:01.975472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:02.011082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 836/1651] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:02.011887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:02.013966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:02.024597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:02.028543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:02.046573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808a5d4-8a21-4c84-8155-473727b0fb09 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:02.060336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:02.166618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 816/1652] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:02.173315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:02.173997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:02.174289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:02.175192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:02.183457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:02.185675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:02.187701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:02.199025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 837/1653] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:02.229897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aede7b3-5ab8-4da2-bd74-8d4749572437 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:02.240780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df89c31-14a6-4659-851f-63ead7586cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:02.241716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df89c31-14a6-4659-851f-63ead7586cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:02.243239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df89c31-14a6-4659-851f-63ead7586cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:02.243239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df89c31-14a6-4659-851f-63ead7586cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:02.268000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df89c31-14a6-4659-851f-63ead7586cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:02.269749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df89c31-14a6-4659-851f-63ead7586cd3 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:02.298116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 817/1654] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:03.143565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:03.146471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:03.189924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a348a5-663a-4ebe-92b8-8842250a6310 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:03.211415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:03.212304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.212588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.212865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] PATH_INFO: `/v3/projects/3408ff2ea39d4c428a122846f19a6809` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.217997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3408ff2ea39d4c428a122846f19a6809)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:03.220669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:03.359618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3408ff2ea39d4c428a122846f19a6809', 'request_id': 'req-80439126-7909-487e-bf38-ce87b701afb6'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.366876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed5f45-e5a0-4cf0-8e19-fd1896a879e9 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.367394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed5f45-e5a0-4cf0-8e19-fd1896a879e9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.367873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed5f45-e5a0-4cf0-8e19-fd1896a879e9 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.374697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3408ff2ea39d4c428a122846f19a6809', 'request_id': 'req-80439126-7909-487e-bf38-ce87b701afb6'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.375321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 818/1655] 10.222.0.61 () {54 vars in 960 bytes} [Tue Jun 7 13:18:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:18:03.381140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.381915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.382354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.387025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3408ff2ea39d4c428a122846f19a6809', 'request_id': 'req-80439126-7909-487e-bf38-ce87b701afb6'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.412335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] The token cache is being invalidate because project 3408ff2ea39d4c428a122846f19a6809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:03.412855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.423108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-80439126-7909-487e-bf38-ce87b701afb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.429398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 838/1656] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:03 2022] DELETE /identity/v3/projects/3408ff2ea39d4c428a122846f19a6809 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:03.440982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.440982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.440982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.463462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:03.463462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:03.475939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:03.479542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:03.561097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:03.638784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 819/1657] 10.222.0.61 () {58 vars in 1057 bytes} [Tue Jun 7 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:03.639503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:03.641637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:03.689763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 839/1658] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:03.690896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:03.710552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916d856-3402-41d7-b90e-a449f321a68d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:03.729794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:03.730750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.731376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.731852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.734910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:03.763570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:03.765387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64046d26-d6e6-4dc7-ba36-6e61da0be802 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:03.774671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9d823-f6ec-4957-8261-6252711178b4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:03.794048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:03.795521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.796499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.797944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] PATH_INFO: `/v3/users/f9a3f501edab44559d30080706179006` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.800556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a3f501edab44559d30080706179006)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:03.802681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 840/1659] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:03.804181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.804481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.804763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.819132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:03.859677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:03.862360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:03.877434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 841/1660] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:03.882330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:03.904273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad36c6-6e8b-43ed-9323-7a08f95c5963 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:03.925349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:03.926245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:03.928883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:03.929286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:03.932878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:03.933297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:03.935908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:03.964032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 842/1661] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:03.965216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:03.974764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a3f501edab44559d30080706179006'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.976518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a3f501edab44559d30080706179006'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.981810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:03.993749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a3f501edab44559d30080706179006'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.995760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a3f501edab44559d30080706179006'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:03.996995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10087707-2326-4592-8b26-196634a05e3e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:04.014264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:04.014264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.014264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.014264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.015238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:04.017323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:04.019573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7945b611-f4a3-4360-9239-99b36b67131e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a3f501edab44559d30080706179006'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:04.042885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 820/1662] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:03 2022] DELETE /identity/v3/users/f9a3f501edab44559d30080706179006 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:04.047456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.047755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.048081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.057597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 843/1663] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:03 2022] POST /identity/v3/projects => generated 393 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:04.063065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:04.066270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:04.075990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:04.077398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:04.224318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:04.265143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 821/1664] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:04.296858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde27a13-5165-4379-8c94-82fcfaed7378 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:04.303569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:04.305512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:04.316028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:04.327424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.328687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.338635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.345335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:04.363637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:04.508275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff77870-394d-4ac5-85be-3e7282e8baf7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:04.522594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 844/1665] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:04 2022] POST /identity/v3/users => generated 297 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:04.524155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:04.527875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:04.528661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.528838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.529258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.564926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adabe7-ffa9-4447-a95e-4a1a48332dcb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:04.570718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:04.572503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:04.612300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:04.615603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.617077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.618018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.624342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:04.632116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 822/1666] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:04.637376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:04.688278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:04.690886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 845/1667] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:04.710321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96d553-e31e-4fef-b323-87c7a906d75a service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:04.728259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc86035-2ad2-453c-822f-e2b90996c4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:04.729051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86035-2ad2-453c-822f-e2b90996c4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.729226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86035-2ad2-453c-822f-e2b90996c4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.729503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc86035-2ad2-453c-822f-e2b90996c4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.730280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86035-2ad2-453c-822f-e2b90996c4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:04.732175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc86035-2ad2-453c-822f-e2b90996c4d9 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:04.747812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 823/1668] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:04.752761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:04.779194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a037b-f965-48ec-85e5-0e171689d7e4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:04.785764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:04.787752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.787882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.788158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] PATH_INFO: `/v3/projects/41dddcaf269640ea80f17bf4beff0fc0/users/0e390e60a38545b3bab7d657b5f36601/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.808781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dddcaf269640ea80f17bf4beff0fc0, user_id=0e390e60a38545b3bab7d657b5f36601, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:04.812230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:04.853064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 846/1669] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:04 2022] PUT /identity/v3/projects/41dddcaf269640ea80f17bf4beff0fc0/users/0e390e60a38545b3bab7d657b5f36601/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:04.855244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:04.855537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:04.855818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:04.895043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] MFA Rules not processed for user `0e390e60a38545b3bab7d657b5f36601`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:04.899366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:05.021493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 824/1670] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:05.027240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:05.241158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938c39-cb51-44e3-8826-baf4e2042cbb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:05.253769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:05.255945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:05.257464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:05.257918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:05.259274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:05.264284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:05.338250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:05.344854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 847/1671] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:05 2022] POST /identity/v3/projects => generated 391 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:05.356758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72df72-9781-4f91-8d6f-7a2606db6afc None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:05.528406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:05.529791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:05.530202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:05.531123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:05.532111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:05.534229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:05.568530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 825/1672] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:05 2022] POST /identity/v3/users => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:05.570032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:05.585168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13c84a-bb3c-4016-a9b5-f27d5b1542ac admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:05.590493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:05.591413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:05.592019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:05.592476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:05.593807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:05.595867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:05.626705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 848/1673] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:05.627956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:05.635151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:05.643309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e2970-f231-4471-ad98-3242ce93fc0d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:05.658899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:05.659449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:05.659729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:05.660006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] PATH_INFO: `/v3/projects/0017e331b477404ab4c3e1594c603c47/users/d23197f2cbf64b72b3bd4f81cda89465/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:05.697196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0017e331b477404ab4c3e1594c603c47, user_id=d23197f2cbf64b72b3bd4f81cda89465, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:05.699952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:05.729742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 826/1674] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:05 2022] PUT /identity/v3/projects/0017e331b477404ab4c3e1594c603c47/users/d23197f2cbf64b72b3bd4f81cda89465/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:05.731574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:05.764281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883aae2-cb63-47ba-8051-da30e8cc2ea1 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:05.784862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:05.903157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c8cd6-3889-46ca-a809-ad9c468ca152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:05.904082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c8cd6-3889-46ca-a809-ad9c468ca152 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:05.904464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c8cd6-3889-46ca-a809-ad9c468ca152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:05.904849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c8cd6-3889-46ca-a809-ad9c468ca152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:05.905680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c8cd6-3889-46ca-a809-ad9c468ca152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:05.907731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c8cd6-3889-46ca-a809-ad9c468ca152 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:05.922871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 849/1675] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:05.926259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:05.945260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0aca4-735c-4a3d-8afc-349100ffd56f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:05.950148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:05.951111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:05.960402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:05.960706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] PATH_INFO: `/v3/projects/0017e331b477404ab4c3e1594c603c47/users/d23197f2cbf64b72b3bd4f81cda89465/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:05.967909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0017e331b477404ab4c3e1594c603c47, user_id=d23197f2cbf64b72b3bd4f81cda89465, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:05.978016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:06.014056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 827/1676] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:05 2022] PUT /identity/v3/projects/0017e331b477404ab4c3e1594c603c47/users/d23197f2cbf64b72b3bd4f81cda89465/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:06.027358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:06.027960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:06.028543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:06.071092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] MFA Rules not processed for user `d23197f2cbf64b72b3bd4f81cda89465`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:06.072776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:06.205127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 850/1677] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:06.224891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:06.228660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:06.267471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:06.272894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:06.459164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d7e8e-b7d0-4405-b20b-f5cba9d2bb2d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:06.476125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:06.478925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:06.479393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:06.479875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] PATH_INFO: `/v3/projects/56cf49983d984317b26f02dc0502e674` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:06.481780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56cf49983d984317b26f02dc0502e674)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:06.484319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:06.507566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601f9d8-2a1d-4382-b759-07bfed02c38c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:06.522688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80424d-7cad-40b6-a682-562e034eecc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:06.523659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80424d-7cad-40b6-a682-562e034eecc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:06.523775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80424d-7cad-40b6-a682-562e034eecc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:06.523973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80424d-7cad-40b6-a682-562e034eecc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:06.557502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80424d-7cad-40b6-a682-562e034eecc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:06.562140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80424d-7cad-40b6-a682-562e034eecc5 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:06.570618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:06.591481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56cf49983d984317b26f02dc0502e674', 'request_id': 'req-633c7722-6ef6-42e3-b9b2-872d095451f8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:06.594675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 828/1678] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:06.613545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56cf49983d984317b26f02dc0502e674', 'request_id': 'req-633c7722-6ef6-42e3-b9b2-872d095451f8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:06.621912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56cf49983d984317b26f02dc0502e674', 'request_id': 'req-633c7722-6ef6-42e3-b9b2-872d095451f8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:06.645678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] The token cache is being invalidate because project 56cf49983d984317b26f02dc0502e674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:06.648098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:06.650332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:06.656609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 851/1679] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:06 2022] DELETE /identity/v3/projects/56cf49983d984317b26f02dc0502e674 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:06.659565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33873-6409-4d6b-9f5f-75b3aea61220 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:06.659969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33873-6409-4d6b-9f5f-75b3aea61220 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:06.660379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33873-6409-4d6b-9f5f-75b3aea61220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:06.697355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e1d33873-6409-4d6b-9f5f-75b3aea61220 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:06.700199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e1d33873-6409-4d6b-9f5f-75b3aea61220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:06.732596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:06.735137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:06.812923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c7722-6ef6-42e3-b9b2-872d095451f8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:06.819690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:06.834172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:06.835987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:06.836431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:06.844757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:06.916650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 829/1680] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:06.918745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:06.919385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:06.919773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:06.972917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:06.982638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:07.007630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 830/1681] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:07.009024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:07.027673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85d52a-0c4d-41e6-83c1-52323f40be47 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:07.043671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:07.044863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.044863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.045224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:07.046136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:07.046578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.048340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.052135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.055940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.089466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 831/1682] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:07.091667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:07.093629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 852/1683] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:07.118244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00796d5c-4cf5-40bd-9bc7-edfa1e772010 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:07.132395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:07.133820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.134253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.135057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:07.136499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.142445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.211857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 832/1684] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:07 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:07.213196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:07.245424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012351-5852-4bed-b583-2b9a7423ad32 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:07.458783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:07.461164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.462517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.463670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:07.465645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.471303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.544426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 853/1685] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:07 2022] POST /identity/v3/users => generated 285 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:07.545748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:07.560548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db073b57-396f-404a-a8c0-b4cd036492a4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:07.574010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:07.578013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.578322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.580925 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:07.581718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.591081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.616769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 833/1686] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:07.618026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:07.634467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:07.647612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d57a19-35af-41fc-a021-ff020f13ef4d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:07.676052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7a9e2-da81-45ff-bf37-b19f3449bd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:07.696469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7a9e2-da81-45ff-bf37-b19f3449bd56 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.698966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7a9e2-da81-45ff-bf37-b19f3449bd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.701233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7a9e2-da81-45ff-bf37-b19f3449bd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:07.704806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7a9e2-da81-45ff-bf37-b19f3449bd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.715351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7a9e2-da81-45ff-bf37-b19f3449bd56 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.763526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 854/1687] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:07.767635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:07.788076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203fd72-deeb-4499-8c1d-a44758772fc8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:07.814423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:07.816338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.817462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.818561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] PATH_INFO: `/v3/projects/acc97c8a5fe94ebbade0d21038e9d5f7/users/540060ad10b74be4b02e235d194634d2/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:07.867356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:07.874751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc97c8a5fe94ebbade0d21038e9d5f7, user_id=540060ad10b74be4b02e235d194634d2, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:07.888092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:07.933978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 834/1688] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:07 2022] PUT /identity/v3/projects/acc97c8a5fe94ebbade0d21038e9d5f7/users/540060ad10b74be4b02e235d194634d2/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:07.948592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:07.948592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:07.948592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:08.076434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] MFA Rules not processed for user `540060ad10b74be4b02e235d194634d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:08.087321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:08.256222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 855/1689] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:08.263171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:08.427756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7db43f-0180-42dd-b51f-0ec44ce3a6c5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:08.434889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:08.435833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:08.437095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:08.437409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:08.440661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:08.441530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:08.486597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 835/1690] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:08 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:08.487382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:08.520741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f4da1-a56c-4753-938c-fc7f7bfc2b87 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:08.699612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:08.700441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:08.700734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:08.701011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:08.701902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:08.706994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:08.766555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 856/1691] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:08 2022] POST /identity/v3/users => generated 290 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:08.770614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:08.787510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf5887-a425-4c6f-bcff-29e7e3aaa6c7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:08.800372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:08.802156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:08.802156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:08.802156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:08.802863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:08.804561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:08.849743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 836/1692] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:08.850082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:08.865360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc56bb-7071-4319-abb9-dd1ca7759f23 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:08.885919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de04718-8c81-447a-844a-ba797492b00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:08.886868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de04718-8c81-447a-844a-ba797492b00e admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:08.887151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de04718-8c81-447a-844a-ba797492b00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:08.887430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de04718-8c81-447a-844a-ba797492b00e admin admin] PATH_INFO: `/v3/projects/d0f857022c1c4c77870f8f9499ca3580/users/a22432254ce14240957d55037f122652/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:08.920304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de04718-8c81-447a-844a-ba797492b00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f857022c1c4c77870f8f9499ca3580, user_id=a22432254ce14240957d55037f122652, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:08.923063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de04718-8c81-447a-844a-ba797492b00e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:08.961791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 857/1693] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:08 2022] PUT /identity/v3/projects/d0f857022c1c4c77870f8f9499ca3580/users/a22432254ce14240957d55037f122652/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:08.966355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:08.980922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99162063-573c-4bae-81e5-7284d5cddf9e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:09.073880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.074167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.074440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.123473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] MFA Rules not processed for user `49c60c4956f8479fbfe93f9f85a8d497`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:09.124708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:09.130523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:09.131233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.131508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.132168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.132504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:09.136531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:09.186164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 837/1694] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:09.186733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:09.213575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8117f4-84e8-4196-85bd-95b7093b4f03 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:09.230033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc51e3-3319-4302-8c32-29cbf9bc0980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:09.230972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc51e3-3319-4302-8c32-29cbf9bc0980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.231259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc51e3-3319-4302-8c32-29cbf9bc0980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.231535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc51e3-3319-4302-8c32-29cbf9bc0980 admin admin] PATH_INFO: `/v3/projects/d0f857022c1c4c77870f8f9499ca3580/users/a22432254ce14240957d55037f122652/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.234955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc51e3-3319-4302-8c32-29cbf9bc0980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f857022c1c4c77870f8f9499ca3580, user_id=a22432254ce14240957d55037f122652, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:09.237720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc51e3-3319-4302-8c32-29cbf9bc0980 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:09.248114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:09.263944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 838/1695] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:09 2022] PUT /identity/v3/projects/d0f857022c1c4c77870f8f9499ca3580/users/a22432254ce14240957d55037f122652/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:09.276153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.277898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.278178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.318959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 858/1696] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:09.329885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:09.333104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:09.352108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] MFA Rules not processed for user `a22432254ce14240957d55037f122652`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:09.354498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:09.470865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:09.523917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3bd2c-4ab4-4ada-ac5d-8b61ddfdbf65 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:09.569383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 839/1697] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:09.582342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:09.626778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:09.627476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.628146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.628445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.680867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee96a01-8c29-4108-baad-a710bea90c7f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:09.689749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e5fe2-fdc7-4957-8f42-885ef46b89aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:09.690427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e5fe2-fdc7-4957-8f42-885ef46b89aa service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.690720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e5fe2-fdc7-4957-8f42-885ef46b89aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.691016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e5fe2-fdc7-4957-8f42-885ef46b89aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.705166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e5fe2-fdc7-4957-8f42-885ef46b89aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:09.718487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e5fe2-fdc7-4957-8f42-885ef46b89aa service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:09.735174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 840/1698] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:09.736751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec3053-ae06-4ced-b9c7-689ccc42f0b3 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.737754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec3053-ae06-4ced-b9c7-689ccc42f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.738172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec3053-ae06-4ced-b9c7-689ccc42f0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.774664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a9ec3053-ae06-4ced-b9c7-689ccc42f0b3 None None] MFA Rules not processed for user `dcf5286e9ca14f56905e6e247e411896`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:09.775515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:09.775598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a9ec3053-ae06-4ced-b9c7-689ccc42f0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:09.776441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:09.810491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 859/1699] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:09.811473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:09.812923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:09.827599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5df79-67f0-4521-8314-ce6fa7c807c9 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:09.831754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:09.832409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.832755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.834078 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.848679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:09.935665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 841/1700] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:09.938811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa75e-c058-4aa9-9a13-6363af12fd99 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:09.939193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa75e-c058-4aa9-9a13-6363af12fd99 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:09.939536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa75e-c058-4aa9-9a13-6363af12fd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:09.977794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cb7fa75e-c058-4aa9-9a13-6363af12fd99 None None] MFA Rules not processed for user `dcf5286e9ca14f56905e6e247e411896`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:09.979804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:09.981075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cb7fa75e-c058-4aa9-9a13-6363af12fd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:09.982941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:10.001763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 860/1701] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:10.002577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 842/1702] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:10.018506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:10.022176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:10.053672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ada4dc-7294-433a-9dfc-cfd6087022c2 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:10.066924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda3a14-4c0c-4d29-b024-6c2fc95516fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:10.068327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3a14-4c0c-4d29-b024-6c2fc95516fb service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:10.068782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3a14-4c0c-4d29-b024-6c2fc95516fb service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:10.069116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda3a14-4c0c-4d29-b024-6c2fc95516fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:10.079424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda3a14-4c0c-4d29-b024-6c2fc95516fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:10.081618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda3a14-4c0c-4d29-b024-6c2fc95516fb service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:10.126618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 861/1703] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:10.677449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12747-a285-4945-990a-1375ec83a922 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:10.677764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12747-a285-4945-990a-1375ec83a922 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:10.678105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe12747-a285-4945-990a-1375ec83a922 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:10.679722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 843/1704] 10.222.0.61 () {54 vars in 912 bytes} [Tue Jun 7 13:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:18:10.684925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b3c8c-6224-4d1a-91c2-6dc08254ffd4 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:10.685393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b3c8c-6224-4d1a-91c2-6dc08254ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:10.685478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b3c8c-6224-4d1a-91c2-6dc08254ffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:10.719293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6f1b3c8c-6224-4d1a-91c2-6dc08254ffd4 None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:10.720811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6f1b3c8c-6224-4d1a-91c2-6dc08254ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:10.879751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 862/1705] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:18:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:10.893103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12747-a285-4945-990a-1375ec83a922 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:10.895584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-dfe12747-a285-4945-990a-1375ec83a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:10.922962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe12747-a285-4945-990a-1375ec83a922 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:10.927956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7c0ac-0583-48a1-8167-048e21d16f19 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:10.928598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7c0ac-0583-48a1-8167-048e21d16f19 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:10.929093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7c0ac-0583-48a1-8167-048e21d16f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:10.932631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:10.933434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:10.933717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:10.934193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:10.943245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:10.945277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:10.948812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-27d7c0ac-0583-48a1-8167-048e21d16f19 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:10.950363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-27d7c0ac-0583-48a1-8167-048e21d16f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:10.955551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 844/1706] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:11.034393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 863/1707] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:18:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:11.379206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:11.380722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:11.549483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:11.617769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2bdb3-b236-4a62-842d-bd871d70a361 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:11.634525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1a9db-8f32-4e2c-9b53-1c51d3a337a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:11.635291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a9db-8f32-4e2c-9b53-1c51d3a337a8 service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:11.635571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a9db-8f32-4e2c-9b53-1c51d3a337a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:11.635841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1a9db-8f32-4e2c-9b53-1c51d3a337a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:11.657523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1a9db-8f32-4e2c-9b53-1c51d3a337a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:11.659226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1a9db-8f32-4e2c-9b53-1c51d3a337a8 service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:11.716441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 845/1708] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:18:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:18:11.915752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:11.916285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:11.916285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:11.982176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] MFA Rules not processed for user `dcf5286e9ca14f56905e6e247e411896`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:11.993033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:12.041746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 864/1709] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:14.472979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08e8b7-b585-4127-bdd2-81f168ba7d1f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:14.473957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08e8b7-b585-4127-bdd2-81f168ba7d1f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:14.474426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08e8b7-b585-4127-bdd2-81f168ba7d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:14.556765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1b08e8b7-b585-4127-bdd2-81f168ba7d1f None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:14.557996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1b08e8b7-b585-4127-bdd2-81f168ba7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:14.618463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 846/1710] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Jun 7 13:18:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:14.664076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:14.665914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:14.701100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1da606-1f7c-4d16-9f13-0510215cf509 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:14.708447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:14.709374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:14.709814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:14.710303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:14.742427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:14.744397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:14.764146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 865/1711] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:15.645978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:15.647467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:15.647926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:15.699804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] MFA Rules not processed for user `49c60c4956f8479fbfe93f9f85a8d497`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:15.703241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:15.728574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 847/1712] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:16.687776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:16.689779 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:16.864264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:17.014696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154658a-9a17-482e-87c4-5d0268e0cc03 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:17.028700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:17.035328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:17.042808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:17.043252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] PATH_INFO: `/v3/users/20a1930545d84798b7bd7053d05ee970` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:17.045756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a1930545d84798b7bd7053d05ee970)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:17.046910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: WARNING py.warnings [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:18:17.046910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: warnings.warn(msg) Jun 07 13:18:17.046910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: Jun 07 13:18:17.048145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:17.201775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:17.246942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4301b-3013-41fd-89c0-c66e17c2d76f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:17.249463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a1930545d84798b7bd7053d05ee970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.250432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a1930545d84798b7bd7053d05ee970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.272366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:17.273279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:17.273567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:17.273840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] PATH_INFO: `/v3/users/540060ad10b74be4b02e235d194634d2` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:17.277207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a1930545d84798b7bd7053d05ee970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.281416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a1930545d84798b7bd7053d05ee970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.298122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=540060ad10b74be4b02e235d194634d2)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:17.300152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:17.335944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fab159cf-921f-438d-b585-44e0c867db0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a1930545d84798b7bd7053d05ee970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.362808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 866/1713] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:16 2022] DELETE /identity/v3/users/20a1930545d84798b7bd7053d05ee970 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:17.368424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:17.369001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:17.369540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:17.435801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:17.440845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:17.557589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '540060ad10b74be4b02e235d194634d2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.558274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '540060ad10b74be4b02e235d194634d2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.595012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '540060ad10b74be4b02e235d194634d2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.595012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '540060ad10b74be4b02e235d194634d2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.620635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a038fe71-b5bf-4a9b-a522-3e53638cebf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '540060ad10b74be4b02e235d194634d2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:17.682730 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 848/1714] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:17 2022] DELETE /identity/v3/users/540060ad10b74be4b02e235d194634d2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:17.687124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:17.687915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:17.688614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:17.748598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:17.751569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 867/1715] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:17.753227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:17.840611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:17.842138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:18.020716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 849/1716] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:18.100818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed3141-b57b-4606-b505-5844d19d33fa None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:18.118486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:18.127205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:18.272204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:18.363439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f43353-e3a8-4ddb-bb7d-b4b05799a841 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:18.380475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a766a3c-c71f-4afc-abbd-280189f3c97b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:18.382645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a766a3c-c71f-4afc-abbd-280189f3c97b service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:18.384618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a766a3c-c71f-4afc-abbd-280189f3c97b service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:18.387132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a766a3c-c71f-4afc-abbd-280189f3c97b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:18.414461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dafc2-b21f-4cac-b050-aec0f9da5fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:18.415174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dafc2-b21f-4cac-b050-aec0f9da5fdd service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:18.415461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dafc2-b21f-4cac-b050-aec0f9da5fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:18.415738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dafc2-b21f-4cac-b050-aec0f9da5fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:18.434856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a766a3c-c71f-4afc-abbd-280189f3c97b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:18.442696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a766a3c-c71f-4afc-abbd-280189f3c97b service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:18.474808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 850/1717] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:18.481184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dafc2-b21f-4cac-b050-aec0f9da5fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:18.501648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dafc2-b21f-4cac-b050-aec0f9da5fdd service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:18.551838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 868/1718] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:18.640697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2b9ea-62cc-48cd-8e59-2a63975ab4e4 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:18.641303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2b9ea-62cc-48cd-8e59-2a63975ab4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:18.641720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2b9ea-62cc-48cd-8e59-2a63975ab4e4 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:18.643845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 851/1719] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:18:18.679556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:18.679978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:18.680396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:18.749237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:18.754637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:18.926184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 869/1720] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:19.721681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2b9ea-62cc-48cd-8e59-2a63975ab4e4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:19.725074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a4a2b9ea-62cc-48cd-8e59-2a63975ab4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:19.763713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2b9ea-62cc-48cd-8e59-2a63975ab4e4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:19.782738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:19.787430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:19.788470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:19.789305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] PATH_INFO: `/v3/users/0e390e60a38545b3bab7d657b5f36601` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:19.794667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e390e60a38545b3bab7d657b5f36601)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:19.804694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:19.923498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:19.939496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:20.046232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d301057-e07c-4e66-8103-1cc7e3ec0975 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:20.077771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e390e60a38545b3bab7d657b5f36601'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.081772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e390e60a38545b3bab7d657b5f36601'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.092341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:20.105782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:20.109080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:20.110246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:20.153045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:20.159456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e390e60a38545b3bab7d657b5f36601'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.162329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e390e60a38545b3bab7d657b5f36601'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.163017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:20.216206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b8773c31-488a-4fea-9150-982f53817527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e390e60a38545b3bab7d657b5f36601'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.220987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 870/1721] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:20.223783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:20.226815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:20.293737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeba92-5bfa-4bdd-aaa6-860725e63ba1 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:20.315233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:20.316368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:20.319212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:20.319494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] PATH_INFO: `/v3/projects/acc97c8a5fe94ebbade0d21038e9d5f7` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:20.326398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc97c8a5fe94ebbade0d21038e9d5f7)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:20.329032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:20.341937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 852/1722] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:19 2022] DELETE /identity/v3/users/0e390e60a38545b3bab7d657b5f36601 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:20.344099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:20.344704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:20.348937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:20.477397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:20.483195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:20.515180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc97c8a5fe94ebbade0d21038e9d5f7', 'request_id': 'req-3c0e65a6-804d-49d2-92d7-276ae2c43b97'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.548223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc97c8a5fe94ebbade0d21038e9d5f7', 'request_id': 'req-3c0e65a6-804d-49d2-92d7-276ae2c43b97'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.581489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc97c8a5fe94ebbade0d21038e9d5f7', 'request_id': 'req-3c0e65a6-804d-49d2-92d7-276ae2c43b97'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.618271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] The token cache is being invalidate because project acc97c8a5fe94ebbade0d21038e9d5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:20.618581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.619528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.623853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 871/1723] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:20 2022] DELETE /identity/v3/projects/acc97c8a5fe94ebbade0d21038e9d5f7 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:20.624932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:20.628670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:20.678723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:20.796779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 853/1724] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:20.799499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:20.803425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:20.812321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e65a6-804d-49d2-92d7-276ae2c43b97 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:20.825449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5927a-993c-4e84-aac2-604c879379ce None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:20.827260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:20.829494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:20.830034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:20.830493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] PATH_INFO: `/v3/projects/2b82200b81a24815acd040d2e3b1e290` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:20.832077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b82200b81a24815acd040d2e3b1e290)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:20.833463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:20.835545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:20.835986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:20.836098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:20.836513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] PATH_INFO: `/v3/users/a22432254ce14240957d55037f122652` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:20.838568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22432254ce14240957d55037f122652)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:20.841387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:20.922607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:20.939965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22432254ce14240957d55037f122652'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.942929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22432254ce14240957d55037f122652'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.960580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22432254ce14240957d55037f122652'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.962944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22432254ce14240957d55037f122652'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.973039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22432254ce14240957d55037f122652'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:20.975157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b82200b81a24815acd040d2e3b1e290', 'request_id': 'req-a2828d98-ffe8-47af-80e0-9834197c956f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.017900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b82200b81a24815acd040d2e3b1e290', 'request_id': 'req-a2828d98-ffe8-47af-80e0-9834197c956f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.033198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b82200b81a24815acd040d2e3b1e290', 'request_id': 'req-a2828d98-ffe8-47af-80e0-9834197c956f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.039980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 854/1725] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:20 2022] DELETE /identity/v3/users/a22432254ce14240957d55037f122652 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:21.046476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:21.050452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] The token cache is being invalidate because project 2b82200b81a24815acd040d2e3b1e290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:21.051009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.052049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a2828d98-ffe8-47af-80e0-9834197c956f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.053015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:21.056291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 872/1726] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:20 2022] DELETE /identity/v3/projects/2b82200b81a24815acd040d2e3b1e290 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:21.063541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c3e65-9411-4785-97e4-ec2831dcd929 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.064875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c3e65-9411-4785-97e4-ec2831dcd929 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.065353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c3e65-9411-4785-97e4-ec2831dcd929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.107814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-918c3e65-9411-4785-97e4-ec2831dcd929 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:21.112230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-918c3e65-9411-4785-97e4-ec2831dcd929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:21.267881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 873/1727] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:21.271661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.272104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.272403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.305020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6563b-3782-48e8-b7bd-966eec947616 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:21.312935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:21.313659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.313933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.314260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.318028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:21.327991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:21.329547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:21.344364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 874/1728] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:21.354272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:21.371760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:21.373330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:21.399233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfcad68-ab55-45a6-be7c-39fbfe889e69 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:21.420542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 855/1729] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:21.420733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:21.421413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.421701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.423615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.426220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:21.428480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:21.438928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:21.451151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:21.452845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:21.486814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9428-b100-4319-b66d-56df257ecb8d service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:21.496947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 875/1730] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:21.504440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:21.507603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:21.514579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.514896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.515167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] PATH_INFO: `/v3/users/dbc72739a09a4d65a064df2d912226f7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.518610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbc72739a09a4d65a064df2d912226f7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:21.529977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:21.546528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0140bf-75c4-4b91-95fe-304aba7aab97 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:21.568059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:21.571928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.572488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.574362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.576552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:21.582548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:21.675197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 876/1731] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:21 2022] POST /identity/v3/projects => generated 405 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:21.679493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:21.689959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbc72739a09a4d65a064df2d912226f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.692055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:21.692813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbc72739a09a4d65a064df2d912226f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.710615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbc72739a09a4d65a064df2d912226f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.711534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbc72739a09a4d65a064df2d912226f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.737622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0db7a68d-0bab-4126-aa58-0bb041f89569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbc72739a09a4d65a064df2d912226f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:21.781287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 856/1732] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:21 2022] DELETE /identity/v3/users/dbc72739a09a4d65a064df2d912226f7 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:21.794115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:21.794419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:21.794702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:21.835182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:21.838948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:21.858764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376f54e-b20f-49d0-88a8-f67a8efad0a4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:21.986244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:22.074419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 857/1733] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:22.088966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.090023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.090312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.091401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.092622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:22.095529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:22.134872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 877/1734] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:21 2022] POST /identity/v3/users => generated 303 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:22.138212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:22.140337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:22.171633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8acc8b-86f9-45d8-aba6-497388ff9158 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:22.172315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:22.177372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:22.187482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.188326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.189201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.189375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.190555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:22.201084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:22.247997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 858/1735] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:22.259438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:22.277207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35165db-24c1-4b98-b68f-a9a8f3aceffa admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:22.294579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.295309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.295640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.299675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.300464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:22.304714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:22.336011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 859/1736] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:22.336923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:22.350600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aed55e-e550-4bff-9f75-20d35c5738f2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:22.351885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a36c58-09e7-4a99-a1c2-e74f0b7383f8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:22.355116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0db218-55f7-4383-ae10-26f0522dba54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.356417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0db218-55f7-4383-ae10-26f0522dba54 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.356417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0db218-55f7-4383-ae10-26f0522dba54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.356908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0db218-55f7-4383-ae10-26f0522dba54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.357545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.358916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.358916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.359205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] PATH_INFO: `/v3/projects/49793a1d91f045ebb4b30b452ef3268e/users/da6e7dba856f4d4fa6ae4413aa0c697f/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.370322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0db218-55f7-4383-ae10-26f0522dba54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:22.372320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0db218-55f7-4383-ae10-26f0522dba54 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:22.378249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49793a1d91f045ebb4b30b452ef3268e, user_id=da6e7dba856f4d4fa6ae4413aa0c697f, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:22.382653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:22.387502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 878/1737] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:22.399524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 860/1738] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:22 2022] PUT /identity/v3/projects/49793a1d91f045ebb4b30b452ef3268e/users/da6e7dba856f4d4fa6ae4413aa0c697f/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:22.406923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.407340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.407691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.444077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:22.446764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:22.579065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:22.626987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 879/1739] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:22.749260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:22.872934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11b91f-35cb-42d8-acff-ad05088c3bc6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:22.883571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd957d-9728-4913-ab75-d034d4ae9cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.884780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd957d-9728-4913-ab75-d034d4ae9cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.885275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd957d-9728-4913-ab75-d034d4ae9cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.886043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd957d-9728-4913-ab75-d034d4ae9cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:22.910223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd957d-9728-4913-ab75-d034d4ae9cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:22.912439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd957d-9728-4913-ab75-d034d4ae9cd0 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:22.932099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 861/1740] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:22.958638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:22.960642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:22.980787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f3909-a13d-4f97-b508-22c5f26a08dd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:22.992917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:22.993813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:22.994138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:22.994451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] PATH_INFO: `/v3/projects/d0f857022c1c4c77870f8f9499ca3580` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.009352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f857022c1c4c77870f8f9499ca3580)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:23.011353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:23.146689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f857022c1c4c77870f8f9499ca3580', 'request_id': 'req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:23.162177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f857022c1c4c77870f8f9499ca3580', 'request_id': 'req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:23.177836 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f857022c1c4c77870f8f9499ca3580', 'request_id': 'req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:23.223116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] The token cache is being invalidate because project d0f857022c1c4c77870f8f9499ca3580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:23.224085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:23.225614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c0573a6-6780-4a1e-99fe-6e2b4184f8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:23.234808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 880/1741] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:22 2022] DELETE /identity/v3/projects/d0f857022c1c4c77870f8f9499ca3580 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:23.237438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:23.237931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:23.238049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.262088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:23.264665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:23.464531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 862/1742] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:23.465041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:23.465496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:23.465933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.521222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:23.528598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:23.556336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 881/1743] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:23.561426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:23.577671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cf48c-993c-4dbc-bb06-3a24089d5ee7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:23.589921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:23.590961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:23.591439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:23.591827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.592551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:23.592845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:23.594608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:23.638932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:23.653910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 863/1744] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:23.657790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714e280-b931-496b-aefe-d2efc00b5666 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:23.677852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:23.678500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:23.678811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:23.679152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.680016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:23.690585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:23.744578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:23.744855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 882/1745] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:23 2022] POST /identity/v3/projects => generated 409 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:23.759149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe1f46-efa3-4d38-b6de-7bf2eea5c70e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:23.794120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:23.796403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:23.820288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:23.924799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14f5e0-c900-4699-a947-77f698d2992c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:23.926163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:23.927399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:23.927686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:23.927968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.930241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:23.934435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:23.937738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:23.938781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:23.939145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:23.939423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] PATH_INFO: `/v3/projects/41dddcaf269640ea80f17bf4beff0fc0` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:23.946943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] RBAC: Authorizing `identity:delete_project(project_id=41dddcaf269640ea80f17bf4beff0fc0)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:23.948929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:23.985474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 864/1746] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:23 2022] POST /identity/v3/users => generated 305 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:23.986849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:24.015339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bfc3e-ed6f-4891-a44c-d3ed09c2bc3b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:24.029136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:24.041387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:24.041732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:24.042023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:24.042793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:24.059046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:24.081882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:24.088866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 865/1747] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:24.092937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:24.103116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41dddcaf269640ea80f17bf4beff0fc0', 'request_id': 'req-8c46b952-a77c-4881-b2d2-9fadcfa255af'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.118979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41dddcaf269640ea80f17bf4beff0fc0', 'request_id': 'req-8c46b952-a77c-4881-b2d2-9fadcfa255af'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.126656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24ef0-5247-4a36-9603-7dc8ae74fa3d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:24.128586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41dddcaf269640ea80f17bf4beff0fc0', 'request_id': 'req-8c46b952-a77c-4881-b2d2-9fadcfa255af'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.196623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] The token cache is being invalidate because project 41dddcaf269640ea80f17bf4beff0fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:24.197336 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.199967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.209424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 883/1748] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:23 2022] DELETE /identity/v3/projects/41dddcaf269640ea80f17bf4beff0fc0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:24.210372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:24.215678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:24.365755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:24.368275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:24.368859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:24.371786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:24.373802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:24.378559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:24.415322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46b952-a77c-4881-b2d2-9fadcfa255af admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:24.424135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 866/1749] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:24.424722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:24.425900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:24.426440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:24.427493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] PATH_INFO: `/v3/users/d23197f2cbf64b72b3bd4f81cda89465` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:24.427915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:24.432730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23197f2cbf64b72b3bd4f81cda89465)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:24.436493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:24.436675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:24.461762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259efa-53a6-411e-b5e9-0459b349c193 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:24.469280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e25fe2-3619-4972-8d31-3c4846abceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:24.471033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e25fe2-3619-4972-8d31-3c4846abceaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:24.471424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e25fe2-3619-4972-8d31-3c4846abceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:24.471842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e25fe2-3619-4972-8d31-3c4846abceaf admin admin] PATH_INFO: `/v3/projects/f3dcc6e8c1ea4b5ba1c0b5eb66ee071c/users/d13ad78d059749ce8a8e85f512138b4e/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:24.498938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e25fe2-3619-4972-8d31-3c4846abceaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3dcc6e8c1ea4b5ba1c0b5eb66ee071c, user_id=d13ad78d059749ce8a8e85f512138b4e, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:24.503634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e25fe2-3619-4972-8d31-3c4846abceaf admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:24.538302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23197f2cbf64b72b3bd4f81cda89465'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.539901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23197f2cbf64b72b3bd4f81cda89465'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.549418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 867/1750] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:24 2022] PUT /identity/v3/projects/f3dcc6e8c1ea4b5ba1c0b5eb66ee071c/users/d13ad78d059749ce8a8e85f512138b4e/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:24.555589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:24.556927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:24.558199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:24.561784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23197f2cbf64b72b3bd4f81cda89465'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.563256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23197f2cbf64b72b3bd4f81cda89465'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.614630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23197f2cbf64b72b3bd4f81cda89465'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:24.619355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] MFA Rules not processed for user `d13ad78d059749ce8a8e85f512138b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:24.621337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:24.666618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 884/1751] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:24 2022] DELETE /identity/v3/users/d23197f2cbf64b72b3bd4f81cda89465 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:24.921395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 868/1752] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:24.927085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:24.934818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:25.080397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:25.105940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94d025-7116-40e5-a8b2-aa81a5e24280 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.115236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.116142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.116520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.117340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.118165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.120729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.168655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 885/1753] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:24 2022] POST /identity/v3/projects => generated 407 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:25.172172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:25.178507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:25.239097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:25.241546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:25.457822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f7e4-1014-40d7-b1a5-123fdbdcdf74 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.480995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0981c58-5c50-42f7-9dbe-69e5da37c4e5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.482615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.483543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.483819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.484093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] PATH_INFO: `/v3/projects/28f4c96b72d341609bfb30ec3d20faae` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.492275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f4c96b72d341609bfb30ec3d20faae)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.494180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.498185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.499539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.499993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.500418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.501414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.505215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.551144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 869/1754] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:25 2022] POST /identity/v3/users => generated 310 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:25.553576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:25.568280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eff9b3-e52a-4c19-abe0-eac84316b352 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.576251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.577257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.577693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.578118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.579127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.582352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.616289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 870/1755] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:25.617722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:25.631697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79955-7c5f-4abf-b2b5-c204e1b78243 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.638437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.639453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.639756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.640026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] PATH_INFO: `/v3/projects/b63d1b04401349d2bb1e2bca81d12850/users/6e509f8f3e4946aeacb98b7e72fa029e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.668045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f4c96b72d341609bfb30ec3d20faae', 'request_id': 'req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:25.686468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63d1b04401349d2bb1e2bca81d12850, user_id=6e509f8f3e4946aeacb98b7e72fa029e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.695456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.698922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f4c96b72d341609bfb30ec3d20faae', 'request_id': 'req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:25.718802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f4c96b72d341609bfb30ec3d20faae', 'request_id': 'req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:25.730751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 871/1756] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:25 2022] PUT /identity/v3/projects/b63d1b04401349d2bb1e2bca81d12850/users/6e509f8f3e4946aeacb98b7e72fa029e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:25.733190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:25.733876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] The token cache is being invalidate because project 28f4c96b72d341609bfb30ec3d20faae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:25.734227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:25.735039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:25.736942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 886/1757] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:25 2022] DELETE /identity/v3/projects/28f4c96b72d341609bfb30ec3d20faae => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:25.745620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:25.747569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:25.943065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db6885-becc-4467-9f73-0bfc407810a2 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.958810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.960331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.960626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.961020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a94b1c-2c24-482d-bb49-77bc4384c0f3 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:25.961631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.964127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.971206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:25.973202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:25.974376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:25.975586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] PATH_INFO: `/v3/users/7357dfac388640d9a84ddda45dd18bff` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:25.979307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7357dfac388640d9a84ddda45dd18bff)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:25.984074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.986075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:25.992297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:26.035961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 872/1758] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:26.040756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:26.046514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:26.099290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c487163-4398-4558-9ee4-0dc5b374cb6d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:26.125402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd22a2-28d5-4b8e-adaf-a5526a3e4d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:26.126451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd22a2-28d5-4b8e-adaf-a5526a3e4d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:26.126736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd22a2-28d5-4b8e-adaf-a5526a3e4d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:26.131429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd22a2-28d5-4b8e-adaf-a5526a3e4d59 admin admin] PATH_INFO: `/v3/projects/b63d1b04401349d2bb1e2bca81d12850/users/6e509f8f3e4946aeacb98b7e72fa029e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:26.138565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd22a2-28d5-4b8e-adaf-a5526a3e4d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63d1b04401349d2bb1e2bca81d12850, user_id=6e509f8f3e4946aeacb98b7e72fa029e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:26.146017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd22a2-28d5-4b8e-adaf-a5526a3e4d59 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:26.149074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7357dfac388640d9a84ddda45dd18bff'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:26.149844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7357dfac388640d9a84ddda45dd18bff'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:26.173368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7357dfac388640d9a84ddda45dd18bff'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:26.176215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7357dfac388640d9a84ddda45dd18bff'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:26.183845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 873/1759] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:26 2022] PUT /identity/v3/projects/b63d1b04401349d2bb1e2bca81d12850/users/6e509f8f3e4946aeacb98b7e72fa029e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:26.194871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:26.195210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:26.195566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:26.233305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7357dfac388640d9a84ddda45dd18bff'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:26.269878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 887/1760] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:25 2022] DELETE /identity/v3/users/7357dfac388640d9a84ddda45dd18bff => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:26.280741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] MFA Rules not processed for user `6e509f8f3e4946aeacb98b7e72fa029e`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:26.284980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:26.350665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:26.437207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 874/1761] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:26.449385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:26.451171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:26.546189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb89bd-5f84-4914-b1df-766c2f4eaeb8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:26.558529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:26.569162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:26.569162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:26.569162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:26.579745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:26.582104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:26.594209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 888/1762] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:26 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:26.867854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:26.881970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcc1db-276f-4766-b88c-bbff865d76a1 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:26.893369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:26.894148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:26.894598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:26.895510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:26.900106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.017320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:27.018818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb356ea-81f3-4a74-bdca-c2ba7718c082 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:27.049927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 875/1763] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:27.099169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.101570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.227348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59838f96-55f8-4f17-b9c8-8726f0860e8c service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:27.236692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:27.237526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.237910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.238276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] PATH_INFO: `/v3/projects/0017e331b477404ab4c3e1594c603c47` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.240500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0017e331b477404ab4c3e1594c603c47)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:27.243052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:27.314542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:27.367616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0017e331b477404ab4c3e1594c603c47', 'request_id': 'req-9f8963b3-5d20-469b-b089-9fcfce785c76'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.379562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0017e331b477404ab4c3e1594c603c47', 'request_id': 'req-9f8963b3-5d20-469b-b089-9fcfce785c76'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.394610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0017e331b477404ab4c3e1594c603c47', 'request_id': 'req-9f8963b3-5d20-469b-b089-9fcfce785c76'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.416735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] The token cache is being invalidate because project 0017e331b477404ab4c3e1594c603c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:27.417182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.417960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.419879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 889/1764] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:27 2022] DELETE /identity/v3/projects/0017e331b477404ab4c3e1594c603c47 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:27.425451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b7e72-eb7b-434b-afe6-bcdfc40a008d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.427463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b7e72-eb7b-434b-afe6-bcdfc40a008d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.427985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b7e72-eb7b-434b-afe6-bcdfc40a008d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.464286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4a3b7e72-eb7b-434b-afe6-bcdfc40a008d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:27.467395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4a3b7e72-eb7b-434b-afe6-bcdfc40a008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.549141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.552489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.651159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 876/1765] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:27.658756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.659466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.659902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.683362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:27.684753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.696593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 877/1766] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:27.700603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.712813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8963b3-5d20-469b-b089-9fcfce785c76 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:27.720766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cddd07e-415f-4144-98a0-a1306aba4a6d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:27.723716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:27.724921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.725363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.725882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] PATH_INFO: `/v3/projects/a6431fdebb5e4ba7b978d361a203a504` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.727957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6431fdebb5e4ba7b978d361a203a504)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:27.729484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:27.730255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:27.730489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.730938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.731353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.732456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:27.733123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:27.734873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:27.748763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 878/1767] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:27.763223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.784593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374563d6-b118-4bf8-a007-63702dfabf26 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:27.803970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:27.804641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.804960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.805736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.806436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:27.808677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:27.830568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6431fdebb5e4ba7b978d361a203a504', 'request_id': 'req-1d2022a5-3fd8-4232-8e9c-c9e503f70468'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.846881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6431fdebb5e4ba7b978d361a203a504', 'request_id': 'req-1d2022a5-3fd8-4232-8e9c-c9e503f70468'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.849212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 879/1768] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:27 2022] POST /identity/v3/projects => generated 381 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:27.850192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.856183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6431fdebb5e4ba7b978d361a203a504', 'request_id': 'req-1d2022a5-3fd8-4232-8e9c-c9e503f70468'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.872011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] The token cache is being invalidate because project a6431fdebb5e4ba7b978d361a203a504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:27.872310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.873243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d2022a5-3fd8-4232-8e9c-c9e503f70468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:27.875531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 890/1769] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:27 2022] DELETE /identity/v3/projects/a6431fdebb5e4ba7b978d361a203a504 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:27.885113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:27.886692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef11ae-f8c1-4765-a8c8-5503977d08ec None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.887194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef11ae-f8c1-4765-a8c8-5503977d08ec None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.887576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef11ae-f8c1-4765-a8c8-5503977d08ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.914726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a9ef11ae-f8c1-4765-a8c8-5503977d08ec None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:27.916046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a9ef11ae-f8c1-4765-a8c8-5503977d08ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.931998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 891/1770] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:27.936174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77782d-1530-42d7-a98f-3b0f1dc72e16 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:27.939917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.940366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.940804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.950751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:27.952074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:27.952643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:27.953322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:27.954517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:27.957346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:27.969253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:27.971418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.982867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 892/1771] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:27.985920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.986072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 880/1772] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:27 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:27.991006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:27.992806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:27.997181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588ee1b-30f6-4874-b9ae-fc3c31a6ba8e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.003028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.004126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.004582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.005032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.006468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:28.006985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.009047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.009629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bbdf2-3ef8-484a-88fc-56d6ae3bad25 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.015540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.016210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.016611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.016917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.017659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.020213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.024285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 893/1773] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:28.028500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.033515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 881/1774] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:28.038329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.041981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566692-aabc-4f86-814f-5d95fd2d4704 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.048536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.049550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.049934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18abd6-a3b9-4757-aab2-aec08b8594b0 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.050154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.050867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.052419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.054402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.056344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.057018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.057307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.057590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.058323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.060629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.077839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:28.078930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 882/1775] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:28.080918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.093044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 894/1776] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:28.096880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.097294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f304d7-8c1f-41e3-a6dc-2a898dbaa7ae admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.196139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f73f51-7243-4b92-bff7-7cdec5ff9af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.198155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f73f51-7243-4b92-bff7-7cdec5ff9af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.198819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f73f51-7243-4b92-bff7-7cdec5ff9af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.199451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f73f51-7243-4b92-bff7-7cdec5ff9af0 admin admin] PATH_INFO: `/v3/projects/26e29d56dda84275b80a36424f352a2f/users/acd136a366ca41d5a1e07aa3b88dd190/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.211914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5557e-a40a-4500-91b7-0c032df9f4e2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.218034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.218716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.219361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.219650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.220719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.222567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f73f51-7243-4b92-bff7-7cdec5ff9af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e29d56dda84275b80a36424f352a2f, user_id=acd136a366ca41d5a1e07aa3b88dd190, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.222847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.227526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f73f51-7243-4b92-bff7-7cdec5ff9af0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.248869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 895/1777] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:28.261322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.261545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 883/1778] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:28 2022] PUT /identity/v3/projects/26e29d56dda84275b80a36424f352a2f/users/acd136a366ca41d5a1e07aa3b88dd190/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:28.264621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.265516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.265802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.277117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81ec8f-7d50-4515-92f0-fa31288951c9 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.310371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] MFA Rules not processed for user `acd136a366ca41d5a1e07aa3b88dd190`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:28.311898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:28.446312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 884/1779] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:28.452220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.463391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:28.497657 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36db2d-87a2-4d8d-a9db-4bedce7b43d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.498347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36db2d-87a2-4d8d-a9db-4bedce7b43d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.498626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36db2d-87a2-4d8d-a9db-4bedce7b43d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.498927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36db2d-87a2-4d8d-a9db-4bedce7b43d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.500807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36db2d-87a2-4d8d-a9db-4bedce7b43d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.503862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36db2d-87a2-4d8d-a9db-4bedce7b43d8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.534343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 896/1780] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:28.538220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6f9fc-bbf7-41ba-8202-2073705b7dac None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.538523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6f9fc-bbf7-41ba-8202-2073705b7dac None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.538806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6f9fc-bbf7-41ba-8202-2073705b7dac None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.540240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 897/1781] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:18:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:18:28.541544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.541665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.546992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.580113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af406dac-6462-4e81-b559-dd00c3c5ab35 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.587421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.587421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.587421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.587867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.588395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.590559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.634740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 885/1782] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:28.636430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.648727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] MFA Rules not processed for user `da6e7dba856f4d4fa6ae4413aa0c697f`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:28.650387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:28.733930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b4fc4-279e-4533-b162-37255c5cb558 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.739876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f71288-1c2e-472a-bca9-52ec8b9bade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.740848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71288-1c2e-472a-bca9-52ec8b9bade6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.741133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71288-1c2e-472a-bca9-52ec8b9bade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.741407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f71288-1c2e-472a-bca9-52ec8b9bade6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.742918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71288-1c2e-472a-bca9-52ec8b9bade6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.744842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f71288-1c2e-472a-bca9-52ec8b9bade6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.763059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 886/1783] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:28.764653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.764970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.765309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.794419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 898/1784] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:28.795303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.798597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:28.803797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:28.809439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963a70-7cc9-44a7-84fd-bf31eb09e6a6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.817927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.818791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.819257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.819931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.821654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.825812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.847128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:28.874485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 899/1785] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/users => generated 298 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:28.875741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.891113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0861c0-6dbf-496d-833e-77fe28025114 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:28.907769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:28.908682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:28.908963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:28.909248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] PATH_INFO: `/v3/projects/be76495f34d24a94a606ad39ccca227c/users/5bc0b544d28e422ebe49e306f1ec8038/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:28.950537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=be76495f34d24a94a606ad39ccca227c, user_id=5bc0b544d28e422ebe49e306f1ec8038, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:28.961266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:28.970873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 887/1786] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:28.971773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:28.973414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:28.992470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 900/1787] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:28 2022] PUT /identity/v3/projects/be76495f34d24a94a606ad39ccca227c/users/5bc0b544d28e422ebe49e306f1ec8038/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:28.993003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.012163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb090b6-8440-4748-a328-c0802622aa7b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.049508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:29.133893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.134873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.135708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.136087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.136908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.142691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.153970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c23fca-0a8e-4419-8bed-c8347ae46daa admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.159750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597578d-ee7e-4c72-aa63-4995c3e4e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.160571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597578d-ee7e-4c72-aa63-4995c3e4e94a admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.160999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597578d-ee7e-4c72-aa63-4995c3e4e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.161375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597578d-ee7e-4c72-aa63-4995c3e4e94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.162224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597578d-ee7e-4c72-aa63-4995c3e4e94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.164269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597578d-ee7e-4c72-aa63-4995c3e4e94a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.179634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 901/1788] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:29.179634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.179634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.179634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.183191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 888/1789] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:28 2022] POST /identity/v3/projects => generated 405 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:29.184166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.185869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:29.215397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] MFA Rules not processed for user `5bc0b544d28e422ebe49e306f1ec8038`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:29.216800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:29.291102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddabbd-ebf2-4919-a2f2-fa3d427bf60f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.297140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.298347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.298643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.298955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.303375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 902/1790] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:29.304513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.313741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.315535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.328880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 889/1791] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:29.329574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.403742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe53dc-b105-433f-b43c-6de02e8da469 service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.414741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.415543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.415722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.415997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.416698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.419846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.443996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:29.459877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2e7f8-c0d9-490c-9645-f17b2abbd6bc None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.477288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 890/1792] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:29 2022] POST /identity/v3/users => generated 309 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:29.477288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.477911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.477911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.479290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.479608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] PATH_INFO: `/v3/projects/c9b230ba239e42eeafe775dcff3387c1/users/7cc430670b7c4166bc79580fd116ce65/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.495425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625f010-b991-468b-9f38-53b1343d4264 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.499167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b230ba239e42eeafe775dcff3387c1, user_id=7cc430670b7c4166bc79580fd116ce65, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.502224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.506756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.507466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.507829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.508043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.508831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.511017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.526656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 903/1793] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:29 2022] PUT /identity/v3/projects/c9b230ba239e42eeafe775dcff3387c1/users/7cc430670b7c4166bc79580fd116ce65/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:29.527873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.535323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:29.559544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 891/1794] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:29 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:29.560591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.598961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888178b-3e70-4203-ab09-e364cdc63273 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.666733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569a000-7883-4310-b143-f74419d81672 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.672609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.673286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.673884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.674208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.675303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.675860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.677200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.677579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.678095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.678347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.679248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.680940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.695469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 892/1795] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:29.695469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.703372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 904/1796] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:29.703372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.720447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58689929-e84c-46de-be02-34c5fc4bbd0e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.721334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163af5f-3b11-4547-ac2c-bfbd692eae0c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.727602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1c68c-fdf2-4da7-9c15-a36a2204c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.728443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1c68c-fdf2-4da7-9c15-a36a2204c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.728731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1c68c-fdf2-4da7-9c15-a36a2204c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.729122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1c68c-fdf2-4da7-9c15-a36a2204c4ea admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.729303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.730173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.730468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1c68c-fdf2-4da7-9c15-a36a2204c4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.730576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.730747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] PATH_INFO: `/v3/projects/c9b230ba239e42eeafe775dcff3387c1/users/7cc430670b7c4166bc79580fd116ce65/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.732797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b230ba239e42eeafe775dcff3387c1, user_id=7cc430670b7c4166bc79580fd116ce65, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.733516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1c68c-fdf2-4da7-9c15-a36a2204c4ea admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.735459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.746080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 905/1797] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:29 2022] PUT /identity/v3/projects/c9b230ba239e42eeafe775dcff3387c1/users/7cc430670b7c4166bc79580fd116ce65/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:29.747724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.757816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 893/1798] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:29 2022] POST /identity/v3/users => generated 298 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:29.759917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.760356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.760729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.790947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] MFA Rules not processed for user `7cc430670b7c4166bc79580fd116ce65`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:29.792308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:29.812838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c572d2-8652-4dd5-b6af-0df75b8439e8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.817758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.818572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.818879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.819159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] PATH_INFO: `/v3/projects/bb82b45f4c354ff6adbee1c94a55c1f7/users/3555de8cf4a2453ba043e4c6919f2307/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.833499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:29.837211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb82b45f4c354ff6adbee1c94a55c1f7, user_id=3555de8cf4a2453ba043e4c6919f2307, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.840592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.856174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 906/1799] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:29 2022] PUT /identity/v3/projects/bb82b45f4c354ff6adbee1c94a55c1f7/users/3555de8cf4a2453ba043e4c6919f2307/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:29.859645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.874469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 894/1800] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:29.875336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.885516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87494404-b97d-415a-9ee2-cd962b4d32c1 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.926617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c269ee-b84d-4125-8843-4212f2cea11b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.931234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.931893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.932205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.932482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.933205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.935015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.935434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:29.943303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.943993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.944444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.944721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.945335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 907/1801] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:29.946283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.947012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.947929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:29.948815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:29.959642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 895/1802] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:29.960404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:29.969698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e6ec1-6466-4a55-9162-1fffd870335d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:29.975091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:29.975940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:29.976222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:29.976505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] PATH_INFO: `/v3/projects/06e6899fbf2d444b8ec617d00fcc4617/users/75c4008e4bb440098b4f9025abde744e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:29.991701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e6899fbf2d444b8ec617d00fcc4617, user_id=75c4008e4bb440098b4f9025abde744e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:29.994426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.007937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06686a64-48d1-416b-8c0a-394dd0b8496c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:30.008373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 896/1803] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:29 2022] PUT /identity/v3/projects/06e6899fbf2d444b8ec617d00fcc4617/users/75c4008e4bb440098b4f9025abde744e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:30.009253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:30.073162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:30.074410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.074730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.075190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.084083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291a4d4-f1b7-48fb-b41e-5af60442de21 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:30.090286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:30.091462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.091747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.092022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] PATH_INFO: `/v3/projects/bb82b45f4c354ff6adbee1c94a55c1f7/users/3555de8cf4a2453ba043e4c6919f2307/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.094943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb82b45f4c354ff6adbee1c94a55c1f7, user_id=3555de8cf4a2453ba043e4c6919f2307, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:30.099051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.122914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 897/1804] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:30 2022] PUT /identity/v3/projects/bb82b45f4c354ff6adbee1c94a55c1f7/users/3555de8cf4a2453ba043e4c6919f2307/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:30.124787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:30.151586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:30.157644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:30.159707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.178737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 908/1805] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:30.212766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82721f7-5404-449b-8a0b-11b163c1f90f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:30.218396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75499fb-0b67-4511-8444-9252afaeb71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:30.219598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75499fb-0b67-4511-8444-9252afaeb71c admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.219714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75499fb-0b67-4511-8444-9252afaeb71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.220019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75499fb-0b67-4511-8444-9252afaeb71c admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.221131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75499fb-0b67-4511-8444-9252afaeb71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:30.222914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75499fb-0b67-4511-8444-9252afaeb71c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.235171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 898/1806] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:30.240470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:30.254986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb525bb-97e9-440e-9834-5b58a83c6a0a service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:30.261249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:30.262216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.262661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.262973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] PATH_INFO: `/v3/projects/06e6899fbf2d444b8ec617d00fcc4617/users/75c4008e4bb440098b4f9025abde744e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.264956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e6899fbf2d444b8ec617d00fcc4617, user_id=75c4008e4bb440098b4f9025abde744e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:30.267757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.285203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 909/1807] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:30 2022] PUT /identity/v3/projects/06e6899fbf2d444b8ec617d00fcc4617/users/75c4008e4bb440098b4f9025abde744e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:30.296563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.297139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.297621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.311376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:30.313971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:30.336617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] MFA Rules not processed for user `75c4008e4bb440098b4f9025abde744e`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:30.338276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:30.409757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf2ee-3a2a-45b5-a331-9670ad2bf32e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:30.414289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:30.415036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.415272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.415873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.425432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:30.457313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 899/1808] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:30.471548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:30.486053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0132ca-b923-41cf-8c2e-a42a3f454b64 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:30.492796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203bdf5-f0ea-46dc-b3c2-c486ab6f1a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:30.493633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203bdf5-f0ea-46dc-b3c2-c486ab6f1a1e service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.494014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203bdf5-f0ea-46dc-b3c2-c486ab6f1a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.494495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203bdf5-f0ea-46dc-b3c2-c486ab6f1a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:30.510149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203bdf5-f0ea-46dc-b3c2-c486ab6f1a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:30.511340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:30.511858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1203bdf5-f0ea-46dc-b3c2-c486ab6f1a1e service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.513515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:30.524855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 900/1809] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:30.525833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 910/1810] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:30.956443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ccb94-1e78-48d7-9351-66da1af7b1a5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:30.956740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ccb94-1e78-48d7-9351-66da1af7b1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:30.957025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ccb94-1e78-48d7-9351-66da1af7b1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:31.021352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9d4ccb94-1e78-48d7-9351-66da1af7b1a5 None None] MFA Rules not processed for user `3555de8cf4a2453ba043e4c6919f2307`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:31.023773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9d4ccb94-1e78-48d7-9351-66da1af7b1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:31.152923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 901/1811] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:31.502147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:31.579276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210cfd5-d6a8-470a-84d3-6576adddc590 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:31.584154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:31.585024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:31.585304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:31.585575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] PATH_INFO: `/v3/users/5bc0b544d28e422ebe49e306f1ec8038` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:31.586811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc0b544d28e422ebe49e306f1ec8038)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:31.588824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:31.632541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:31.646696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc0b544d28e422ebe49e306f1ec8038'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:31.647555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc0b544d28e422ebe49e306f1ec8038'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:31.655513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc0b544d28e422ebe49e306f1ec8038'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:31.656165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc0b544d28e422ebe49e306f1ec8038'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:31.668808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-779f4010-2e2b-469d-afa3-3708b1b37bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc0b544d28e422ebe49e306f1ec8038'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:31.682545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 911/1812] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:31 2022] DELETE /identity/v3/users/5bc0b544d28e422ebe49e306f1ec8038 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:31.687151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:31.687584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:31.687758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:31.707167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:31.708425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:31.803057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f34826-56f7-4bb9-9810-b1a992dfbb6c None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:31.803760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f34826-56f7-4bb9-9810-b1a992dfbb6c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:31.804246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f34826-56f7-4bb9-9810-b1a992dfbb6c None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:31.806526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 912/1813] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:18:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:18:31.812539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:31.812927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:31.813274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:31.842604 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:31.845518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:31.854866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 902/1814] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:31.866348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:31.868031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:31.938285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a171e-3129-4514-892a-bb341456db80 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:31.944226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:31.945973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:31.946262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:31.946549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:31.958350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:31.960169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:31.963468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 913/1815] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:18:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:31.966579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:31.972358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 903/1816] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:31.978873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:31.981149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:32.002251 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364680a9-f4fe-4b05-b0a7-2f3b257ef513 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:32.007611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:32.008254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:32.008537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:32.008810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:32.017720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:32.019326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:32.030018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 914/1817] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:32.744594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:32.747237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:32.763922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec15ba-c252-4b3f-82f1-15b91dc1bdd2 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:32.769422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:32.770394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:32.770680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:32.770980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] PATH_INFO: `/v3/users/da6e7dba856f4d4fa6ae4413aa0c697f` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:32.772761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6e7dba856f4d4fa6ae4413aa0c697f)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:32.775013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:32.779616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:32.781356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:32.821714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c51e-61a2-4181-bddc-185734d5b55c service placement] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:32.834686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:32.835756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:32.836049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:32.836392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] PATH_INFO: `/v3/projects/be76495f34d24a94a606ad39ccca227c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:32.841572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be76495f34d24a94a606ad39ccca227c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:32.844009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:32.903559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6e7dba856f4d4fa6ae4413aa0c697f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.905288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6e7dba856f4d4fa6ae4413aa0c697f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.938614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6e7dba856f4d4fa6ae4413aa0c697f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.940550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6e7dba856f4d4fa6ae4413aa0c697f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.954430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6e7dba856f4d4fa6ae4413aa0c697f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.972672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be76495f34d24a94a606ad39ccca227c', 'request_id': 'req-030545d8-a7b9-48cc-b0fd-d742f54b60d7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.989923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be76495f34d24a94a606ad39ccca227c', 'request_id': 'req-030545d8-a7b9-48cc-b0fd-d742f54b60d7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:32.991194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 904/1818] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:32 2022] DELETE /identity/v3/users/da6e7dba856f4d4fa6ae4413aa0c697f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:33.000756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be76495f34d24a94a606ad39ccca227c', 'request_id': 'req-030545d8-a7b9-48cc-b0fd-d742f54b60d7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.035365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] The token cache is being invalidate because project be76495f34d24a94a606ad39ccca227c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:33.036425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.037320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-030545d8-a7b9-48cc-b0fd-d742f54b60d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.039148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 915/1819] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:32 2022] DELETE /identity/v3/projects/be76495f34d24a94a606ad39ccca227c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:33.042924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:33.044360 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:33.124869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc4fadd-e507-4984-a8b6-1cdcb6b84265 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:33.130850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:33.131744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.132285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.132566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] PATH_INFO: `/v3/users/75c4008e4bb440098b4f9025abde744e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.134304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c4008e4bb440098b4f9025abde744e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:33.136310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:33.148449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183265-4671-4c29-ab6c-6a4e276efe61 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.148733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183265-4671-4c29-ab6c-6a4e276efe61 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.149008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183265-4671-4c29-ab6c-6a4e276efe61 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.150498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 916/1820] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 07 13:18:33.164361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.164656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.166784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.204219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:33.205749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:33.225085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c4008e4bb440098b4f9025abde744e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.225852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c4008e4bb440098b4f9025abde744e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.239684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c4008e4bb440098b4f9025abde744e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.240517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c4008e4bb440098b4f9025abde744e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.253201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:33.271495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c4008e4bb440098b4f9025abde744e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.296137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:33.316775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 905/1821] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:33 2022] DELETE /identity/v3/users/75c4008e4bb440098b4f9025abde744e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:33.317915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:33.319483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:33.362772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 917/1822] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:18:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:33.364031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:33.366598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:33.387354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ecb8-6570-4171-9006-da95f6f8a084 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:33.471506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73574b0-76f9-4b10-b825-3044e100ccc5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:33.485595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:33.487188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.489130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.491545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] PATH_INFO: `/v3/projects/49793a1d91f045ebb4b30b452ef3268e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.495961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49793a1d91f045ebb4b30b452ef3268e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:33.496964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: WARNING py.warnings [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 07 13:18:33.496964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: warnings.warn(msg) Jun 07 13:18:33.496964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: Jun 07 13:18:33.500995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:33.506593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:33.507425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.507857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.508371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.521846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:33.523872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:33.549215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 918/1823] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:33.629554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49793a1d91f045ebb4b30b452ef3268e', 'request_id': 'req-d0438d2a-ecb9-4dad-8792-651c7c12e071'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.646002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49793a1d91f045ebb4b30b452ef3268e', 'request_id': 'req-d0438d2a-ecb9-4dad-8792-651c7c12e071'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.656309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49793a1d91f045ebb4b30b452ef3268e', 'request_id': 'req-d0438d2a-ecb9-4dad-8792-651c7c12e071'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.672515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] The token cache is being invalidate because project 49793a1d91f045ebb4b30b452ef3268e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:33.673010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.673827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.676041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 906/1824] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:33 2022] DELETE /identity/v3/projects/49793a1d91f045ebb4b30b452ef3268e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:33.680668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:33.693271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a93fee-6ea8-4439-9b92-38bb1301f1dc service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:33.769847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:33.772184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:33.783237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:33.784301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.785465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.785926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] PATH_INFO: `/v3/users/3555de8cf4a2453ba043e4c6919f2307` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.787511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3555de8cf4a2453ba043e4c6919f2307)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:33.790090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:33.797251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0438d2a-ecb9-4dad-8792-651c7c12e071 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:33.806290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:33.808015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.808015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.808344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] PATH_INFO: `/v3/projects/06e6899fbf2d444b8ec617d00fcc4617` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.812140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e6899fbf2d444b8ec617d00fcc4617)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:33.814091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:33.847485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:33.856888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3555de8cf4a2453ba043e4c6919f2307'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.857991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3555de8cf4a2453ba043e4c6919f2307'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.868992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3555de8cf4a2453ba043e4c6919f2307'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.870573 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3555de8cf4a2453ba043e4c6919f2307'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.872990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e6899fbf2d444b8ec617d00fcc4617', 'request_id': 'req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.880937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e6899fbf2d444b8ec617d00fcc4617', 'request_id': 'req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.886107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7c7a0de5-85f2-4f12-bf0a-668f8dbf425b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3555de8cf4a2453ba043e4c6919f2307'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.897363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e6899fbf2d444b8ec617d00fcc4617', 'request_id': 'req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.901243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 919/1825] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:33 2022] DELETE /identity/v3/users/3555de8cf4a2453ba043e4c6919f2307 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:33.908834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] The token cache is being invalidate because project 06e6899fbf2d444b8ec617d00fcc4617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:33.909188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.909960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e4d5d4e9-6850-4a31-84eb-f2b327d63bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:33.915326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 907/1826] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:33 2022] DELETE /identity/v3/projects/06e6899fbf2d444b8ec617d00fcc4617 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:33.918250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:33.918548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:33.918857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:33.938105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:33.939368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.002576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 920/1827] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:34.011464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.011846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.012129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.019748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:34.031399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:34.032606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.041164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 908/1828] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:34.045292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.056099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fa134-4287-497a-a9cc-26be5ceec63f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.064563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.065246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.065529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.065806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.066545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:34.066870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.068536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.088683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 921/1829] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:34.092327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.093061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.093769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.108402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c17a9-e181-4544-b257-d1131faffd57 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.110997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab29fe-aa08-4160-bbae-ac99e552840f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.113306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.114024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.114899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.115190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.116190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.116877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.117893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.118107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.118476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.118929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] PATH_INFO: `/v3/projects/bb82b45f4c354ff6adbee1c94a55c1f7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.120276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb82b45f4c354ff6adbee1c94a55c1f7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.123421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.150055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 922/1830] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:34.155196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.187496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb82b45f4c354ff6adbee1c94a55c1f7', 'request_id': 'req-9f6774b6-58a6-444a-ada4-bf2722a180cd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.195320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb82b45f4c354ff6adbee1c94a55c1f7', 'request_id': 'req-9f6774b6-58a6-444a-ada4-bf2722a180cd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.204228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb82b45f4c354ff6adbee1c94a55c1f7', 'request_id': 'req-9f6774b6-58a6-444a-ada4-bf2722a180cd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.218488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] The token cache is being invalidate because project bb82b45f4c354ff6adbee1c94a55c1f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:34.218784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.219607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9f6774b6-58a6-444a-ada4-bf2722a180cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.221746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 909/1831] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:34 2022] DELETE /identity/v3/projects/bb82b45f4c354ff6adbee1c94a55c1f7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:34.228389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee58fc-f65d-4d47-ba3a-467b352ab18d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.228684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee58fc-f65d-4d47-ba3a-467b352ab18d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.228964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee58fc-f65d-4d47-ba3a-467b352ab18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.233213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d519800-5a1f-447b-b57b-d2f358550281 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.251638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e9ee58fc-f65d-4d47-ba3a-467b352ab18d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:34.252844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e9ee58fc-f65d-4d47-ba3a-467b352ab18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.258958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:34.295347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.295945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.296213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.296485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.297204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.299124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.314239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 910/1832] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:34.322490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 923/1833] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:34.322635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.322635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.322921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.323492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.325153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.337621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad45a5-746b-4a48-85d6-50ac59709832 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.342727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.342917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:34.343722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.344154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.344313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.344727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.345856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.349078 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.355211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 911/1834] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:34.359523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.367930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 924/1835] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:34.368911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.371702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be567b-1bbf-4679-984f-b0afd84d03a5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.376564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.377226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.377513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.377791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.378457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de546ea6-354c-4358-8511-7ed9817b66e0 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.378567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:34.378852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.380710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.383234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.383869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.384154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.384429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.385144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.386722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.392731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 912/1836] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:18:34.400004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 925/1837] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:34.400135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.400912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.407144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b6e12-6bb6-4d78-8885-866b73ef682c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.410789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda433-d31e-4e03-a486-63acfdaa3810 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.412304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.412938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.413219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.413492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.414193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.416117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.416684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed8b1-020a-4559-874a-7ce99fece55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.417571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed8b1-020a-4559-874a-7ce99fece55f admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.417860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed8b1-020a-4559-874a-7ce99fece55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.418142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed8b1-020a-4559-874a-7ce99fece55f admin admin] PATH_INFO: `/v3/projects/f6a220fd65dc4692a1da7c81be855c9c/users/96257035cd5b436b9daf453bfe81d731/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.435211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed8b1-020a-4559-874a-7ce99fece55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a220fd65dc4692a1da7c81be855c9c, user_id=96257035cd5b436b9daf453bfe81d731, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.438229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed8b1-020a-4559-874a-7ce99fece55f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.448912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 913/1838] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:34.452165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 926/1839] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:34 2022] PUT /identity/v3/projects/f6a220fd65dc4692a1da7c81be855c9c/users/96257035cd5b436b9daf453bfe81d731/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:34.453155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.474770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.475110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.475385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.503508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:34.503794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] MFA Rules not processed for user `96257035cd5b436b9daf453bfe81d731`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:34.504740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.526904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd60d6f-57b0-4a2c-9b52-4e504c154005 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.531558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.532250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.532539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.532817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.533505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.535478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.562881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 914/1840] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:34.566163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.577475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 927/1841] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:34.578741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ae4a9-57af-4c5f-8be6-eb60c17d0f6e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.585564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.586241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.586526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.586804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.587572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.589163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.598712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.600209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.601113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 915/1842] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:34.604649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.614107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f14f08-c43d-4617-98c5-7358e0cf36d8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.618684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.619347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.619631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.619905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.620610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.622168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.632964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 916/1843] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:34.636945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.649842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32075525-398e-47e6-b558-af964ef940ed admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.657415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6458a-e2a2-42e1-b81a-ebb078203000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.658588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6458a-e2a2-42e1-b81a-ebb078203000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.658900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6458a-e2a2-42e1-b81a-ebb078203000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.659180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6458a-e2a2-42e1-b81a-ebb078203000 admin admin] PATH_INFO: `/v3/projects/f1315637655e489b83b4ea9b99b80ac9/users/34fe838276e14fd48113cee570857048/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.662501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15707e-b89c-465a-8b39-2e056369c9f2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.666443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.667185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.667468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.669322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.676306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6458a-e2a2-42e1-b81a-ebb078203000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1315637655e489b83b4ea9b99b80ac9, user_id=34fe838276e14fd48113cee570857048, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.678432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.679249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6458a-e2a2-42e1-b81a-ebb078203000 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.679892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.695210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 917/1844] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:34 2022] PUT /identity/v3/projects/f1315637655e489b83b4ea9b99b80ac9/users/34fe838276e14fd48113cee570857048/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:34.695513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 928/1845] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:34.705169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.705551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.705900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.715533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.723297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:34.734281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff548b-a43a-4e9a-8953-9fa1c50e1892 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.738439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] MFA Rules not processed for user `34fe838276e14fd48113cee570857048`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:34.739760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.799956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.801153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.801438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.801775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] PATH_INFO: `/v3/users/96257035cd5b436b9daf453bfe81d731` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.803063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96257035cd5b436b9daf453bfe81d731)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.805709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.831655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 918/1846] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:34.836796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.850389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd174cf-83fd-4739-8ba3-2fb22d78dfa4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:34.855428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:34.856432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.856822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.857108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.857796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:34.858473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96257035cd5b436b9daf453bfe81d731'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.859213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96257035cd5b436b9daf453bfe81d731'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.859767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:34.866253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96257035cd5b436b9daf453bfe81d731'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.867032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96257035cd5b436b9daf453bfe81d731'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.884022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e686e8fa-e48a-4651-871a-469d6907021f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96257035cd5b436b9daf453bfe81d731'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:34.892466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 919/1847] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:34.896876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:34.898682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.899673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 929/1848] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:34 2022] DELETE /identity/v3/users/96257035cd5b436b9daf453bfe81d731 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:34.903861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:34.904155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:34.904436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:34.922319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:34.923557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:34.973021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:34.999190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84829bbb-5ae7-4aec-8347-9fdd9dd09182 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.009677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 930/1849] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:35.019033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.020426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:35.047392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:35.057224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.057845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.058134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.058409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.059136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.061095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.081891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2e84-7433-4b34-9296-f868cb29b145 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.086549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 920/1850] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:34 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:35.086702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.086702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.086885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.087092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.087601 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.097032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.097515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2015b1-2e7a-467d-8536-1e07e9adf463 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.098541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.102170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.102811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.103124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.103824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.104118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.105702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.108204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 931/1851] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:35.116697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 921/1852] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:35.120756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.130814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62695756-ba43-4e63-b64c-cfd9022b3e7d service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.136946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.137920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.138266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.138611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] PATH_INFO: `/v3/projects/5183629314bd453880a8c93e7954f3ce/users/08e57361303c4bebb1eb3f2a9a84bdd7/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.153636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5183629314bd453880a8c93e7954f3ce, user_id=08e57361303c4bebb1eb3f2a9a84bdd7, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.156466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.168803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 932/1853] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:35 2022] PUT /identity/v3/projects/5183629314bd453880a8c93e7954f3ce/users/08e57361303c4bebb1eb3f2a9a84bdd7/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:35.173118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.183923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1a5de-da73-45a4-9325-6be88e7397b5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.243452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83a7ed-b45f-4cc3-9310-f75f5a2539a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.244123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83a7ed-b45f-4cc3-9310-f75f5a2539a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.244418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83a7ed-b45f-4cc3-9310-f75f5a2539a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.244696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83a7ed-b45f-4cc3-9310-f75f5a2539a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.245417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83a7ed-b45f-4cc3-9310-f75f5a2539a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.247089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83a7ed-b45f-4cc3-9310-f75f5a2539a2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.265436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 922/1854] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:35.270142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.283534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a36d3b-e6c4-4a48-ac2c-23f81217fc11 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.289173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.290163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.290622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.291028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] PATH_INFO: `/v3/projects/5183629314bd453880a8c93e7954f3ce/users/08e57361303c4bebb1eb3f2a9a84bdd7/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.293135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5183629314bd453880a8c93e7954f3ce, user_id=08e57361303c4bebb1eb3f2a9a84bdd7, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.296101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.307713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 933/1855] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:35 2022] PUT /identity/v3/projects/5183629314bd453880a8c93e7954f3ce/users/08e57361303c4bebb1eb3f2a9a84bdd7/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:35.319853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.320150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.320435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.355535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] MFA Rules not processed for user `08e57361303c4bebb1eb3f2a9a84bdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:35.356775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:35.407663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.410039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:35.454354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:35.461392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 923/1856] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:35.472349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.473746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:35.496184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15ebb8-ffc5-4fc3-8813-c6b5cfc22e39 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.500850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.501709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.501991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.502274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] PATH_INFO: `/v3/projects/f6a220fd65dc4692a1da7c81be855c9c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.503480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a220fd65dc4692a1da7c81be855c9c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.505906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.537388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e9849-91e3-4e05-8e41-229fb4ea53b2 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.541347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85c1ac-1975-4b2b-9423-117b6dddbf56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.542000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85c1ac-1975-4b2b-9423-117b6dddbf56 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.542276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85c1ac-1975-4b2b-9423-117b6dddbf56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.542552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85c1ac-1975-4b2b-9423-117b6dddbf56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.551970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85c1ac-1975-4b2b-9423-117b6dddbf56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.553736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85c1ac-1975-4b2b-9423-117b6dddbf56 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.553862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a220fd65dc4692a1da7c81be855c9c', 'request_id': 'req-4f2fb66b-80cb-4154-863b-cd7f9fec8852'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:35.562659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a220fd65dc4692a1da7c81be855c9c', 'request_id': 'req-4f2fb66b-80cb-4154-863b-cd7f9fec8852'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:35.565038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 924/1857] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:35.569224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a220fd65dc4692a1da7c81be855c9c', 'request_id': 'req-4f2fb66b-80cb-4154-863b-cd7f9fec8852'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:35.580557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] The token cache is being invalidate because project f6a220fd65dc4692a1da7c81be855c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:35.580557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:35.581266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f2fb66b-80cb-4154-863b-cd7f9fec8852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:35.587589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 934/1858] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:35 2022] DELETE /identity/v3/projects/f6a220fd65dc4692a1da7c81be855c9c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:35.589366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.589871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.590158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.614196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:35.615459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:35.686478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 925/1859] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:35.694962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.695259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.695544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.714041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:35.715256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:35.725173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 935/1860] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:35.729247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.738944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d374-2021-4c57-b1f9-5fda06a0b0a0 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.744120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.744798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.745066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.745356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.746131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:35.746472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.748229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.767932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 926/1861] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:35.769011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.782900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b23eaf-e8e9-4a40-a059-3d01f4e58d42 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.789570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.790269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.790558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.790857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.791584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.793680 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.804394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:35.823757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 936/1862] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:35 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:35.828276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.839708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b02252-7309-434b-95d1-253f26697143 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.912962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.913679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.914033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.914370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.915179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.917210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.938605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 927/1863] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:35 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:35.943641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.959892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2b67c-9b47-45c5-aac3-96be290bce39 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:35.965908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:35.966665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:35.967068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:35.967417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:35.968228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:35.971272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:35.984082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 937/1864] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:35.988872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.993668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:35.995539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:36.001635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86038c43-1055-4400-aec3-abc17b1fa136 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.006798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.007567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.007925 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.008414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.009155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.010854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.015481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e03e3-1af6-4fdc-a7f9-0c038a0872d6 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.021610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.025250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 928/1865] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:36.025749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.025749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.025749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] PATH_INFO: `/v3/users/34fe838276e14fd48113cee570857048` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.026781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34fe838276e14fd48113cee570857048)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.027249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:36.029753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.041636 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f16c27-cf4f-4736-90d6-eaebb77ccbc3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.047742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbcb8e-1c4d-4ac6-aa44-6445636491ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.048633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbcb8e-1c4d-4ac6-aa44-6445636491ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.048958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbcb8e-1c4d-4ac6-aa44-6445636491ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.049242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbcb8e-1c4d-4ac6-aa44-6445636491ac admin admin] PATH_INFO: `/v3/projects/791eab758a8b4db98e721ce1fbcd46d1/users/f3d9eb831c1f4d3ca450edd756729b06/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.052216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:36.069246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbcb8e-1c4d-4ac6-aa44-6445636491ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=791eab758a8b4db98e721ce1fbcd46d1, user_id=f3d9eb831c1f4d3ca450edd756729b06, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.071946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbcb8e-1c4d-4ac6-aa44-6445636491ac admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.085874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 929/1866] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:36 2022] PUT /identity/v3/projects/791eab758a8b4db98e721ce1fbcd46d1/users/f3d9eb831c1f4d3ca450edd756729b06/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:36.086754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34fe838276e14fd48113cee570857048'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.087599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34fe838276e14fd48113cee570857048'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.096331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34fe838276e14fd48113cee570857048'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.097531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34fe838276e14fd48113cee570857048'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.097801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.098152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.100607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.116263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b5d39970-20e5-46bc-8a18-b5b169f1b233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34fe838276e14fd48113cee570857048'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.131655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 938/1867] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:35 2022] DELETE /identity/v3/users/34fe838276e14fd48113cee570857048 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:36.132366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] MFA Rules not processed for user `f3d9eb831c1f4d3ca450edd756729b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:36.133668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:36.136167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.136462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.136743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.157273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:36.158491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:36.206929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 930/1868] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:36.218075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:36.219562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:36.231297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 939/1869] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:36.241805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:36.243492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:36.280278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7911fa-5d0b-402f-81c9-1b7a522664db None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.285669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.286869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.287159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.287437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.296354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.297802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.306608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c5a867-e799-4219-a4d0-f82fda9b69b9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.308081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 931/1870] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:36.311311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.312397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.312776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.313164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.321656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:36.324412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.326598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.338984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 940/1871] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:36.664465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:36.666003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:36.681202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0427034-f5c5-48be-bec1-57b90f014022 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.698209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.699147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.699438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.699720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] PATH_INFO: `/v3/projects/f1315637655e489b83b4ea9b99b80ac9` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.700928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1315637655e489b83b4ea9b99b80ac9)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.702866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.752194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1315637655e489b83b4ea9b99b80ac9', 'request_id': 'req-a095a8cd-e5c8-4988-a992-733d3ba18cfc'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.758301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1315637655e489b83b4ea9b99b80ac9', 'request_id': 'req-a095a8cd-e5c8-4988-a992-733d3ba18cfc'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.765434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1315637655e489b83b4ea9b99b80ac9', 'request_id': 'req-a095a8cd-e5c8-4988-a992-733d3ba18cfc'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.772239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:36.777896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] The token cache is being invalidate because project f1315637655e489b83b4ea9b99b80ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:36.778373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.779458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.781098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 932/1872] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:36 2022] DELETE /identity/v3/projects/f1315637655e489b83b4ea9b99b80ac9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:36.785059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:36.795685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51decc75-51a5-412b-a8f5-fb79f3125a0f service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:36.851797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:36.852587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:36.852872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:36.853148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] PATH_INFO: `/v3/users/08e57361303c4bebb1eb3f2a9a84bdd7` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:36.854347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e57361303c4bebb1eb3f2a9a84bdd7)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:36.856288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:36.901116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e57361303c4bebb1eb3f2a9a84bdd7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.901754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e57361303c4bebb1eb3f2a9a84bdd7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.908610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e57361303c4bebb1eb3f2a9a84bdd7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.909246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e57361303c4bebb1eb3f2a9a84bdd7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.920817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9ef9d5a0-e7f7-4914-9e9a-7b0981af453a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e57361303c4bebb1eb3f2a9a84bdd7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:36.934213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 941/1873] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:36 2022] DELETE /identity/v3/users/08e57361303c4bebb1eb3f2a9a84bdd7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:37.226784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.228354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:37.244936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095a8cd-e5c8-4988-a992-733d3ba18cfc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.319649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.320515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.320828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.321106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] PATH_INFO: `/v3/projects/5183629314bd453880a8c93e7954f3ce` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.322295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5183629314bd453880a8c93e7954f3ce)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.324270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.371855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5183629314bd453880a8c93e7954f3ce', 'request_id': 'req-320cd7ca-80b2-41f9-9f45-a54225e2a55a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:37.378527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5183629314bd453880a8c93e7954f3ce', 'request_id': 'req-320cd7ca-80b2-41f9-9f45-a54225e2a55a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:37.386679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5183629314bd453880a8c93e7954f3ce', 'request_id': 'req-320cd7ca-80b2-41f9-9f45-a54225e2a55a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:37.397983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] The token cache is being invalidate because project 5183629314bd453880a8c93e7954f3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:37.398293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:37.398947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-320cd7ca-80b2-41f9-9f45-a54225e2a55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:37.404901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 933/1874] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:37 2022] DELETE /identity/v3/projects/5183629314bd453880a8c93e7954f3ce => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:37.410199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.410535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.410814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.432514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:37.434237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:37.494117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 942/1875] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:37.501651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.502185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.502469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.521369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:37.522584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:37.530898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 934/1876] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:37.535052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.544908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b2a1a-6f5f-4149-869b-220607285413 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.549921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.550682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.551069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.551355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.552897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:37.553147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.554808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.566710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 943/1877] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:37.571630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.580843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266bb42-3f28-4771-b8cd-78b48fd2ddcb None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.585467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.586114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.586398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.586667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.587443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.589389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.616472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 935/1878] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:37 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:37.620431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.625639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:37.631903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f056b-b21e-4427-b013-86d2ce69df5b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.693645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.694408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.694760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.695154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.695947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.697984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.722097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 944/1879] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:37 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:37.726365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.736895 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c72312-268b-4279-920b-537fd73e59e8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.743966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.744781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.745076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.745361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.746092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.747739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.759675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 936/1880] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:37.763781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.778632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68147-f845-48a8-905d-5331065ff192 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.784225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879f6f-72ba-4642-8859-3a69aef54dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.784907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879f6f-72ba-4642-8859-3a69aef54dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.785189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879f6f-72ba-4642-8859-3a69aef54dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.785461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879f6f-72ba-4642-8859-3a69aef54dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.786726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81879f6f-72ba-4642-8859-3a69aef54dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.788400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81879f6f-72ba-4642-8859-3a69aef54dc5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.798917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 945/1881] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:37.803419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.821845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f73e45-f407-4217-8e88-b048209039cf admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:37.826490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:37.827392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.827677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.827952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] PATH_INFO: `/v3/projects/82eec3d610b44fc18e9e1f2300e5c953/users/ef351555df3748bbb3d10b7b40396bf3/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.843159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82eec3d610b44fc18e9e1f2300e5c953, user_id=ef351555df3748bbb3d10b7b40396bf3, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:37.846050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:37.859251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 937/1882] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:37 2022] PUT /identity/v3/projects/82eec3d610b44fc18e9e1f2300e5c953/users/ef351555df3748bbb3d10b7b40396bf3/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:37.870856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:37.871321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:37.871430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:37.898634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] MFA Rules not processed for user `ef351555df3748bbb3d10b7b40396bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:37.899971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:37.963681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 946/1883] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:37 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:37.968204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:37.995128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:38.030728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8a58c-107d-4519-a495-67aa4d47f61c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.035337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.036585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.036886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.037149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.037823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.039746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.063607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 938/1884] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:37 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:38.067685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:38.078817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da139b5-a306-4cf8-834a-792ed6522697 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.136138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.136786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.137069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.137342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.138104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.140107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.156752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:38.167153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 947/1885] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:38 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:38.167782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:38.178268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec9899-4abc-4f4f-a66a-4415118017d1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.182810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.183493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.183773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.184046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.184767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.186336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.198861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 939/1886] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:38.201727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:38.212471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8207a-5f42-48d1-b7e4-5d9826c9567a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.217130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.218055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.218337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.218618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] PATH_INFO: `/v3/projects/494e874647ec43b2845ce229b6f571ad/users/eeba0c5ede8d40a5975493d71d22d548/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.234202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=494e874647ec43b2845ce229b6f571ad, user_id=eeba0c5ede8d40a5975493d71d22d548, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.236870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.250360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 948/1887] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:38 2022] PUT /identity/v3/projects/494e874647ec43b2845ce229b6f571ad/users/eeba0c5ede8d40a5975493d71d22d548/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:38.254459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:38.264609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56852866-938a-40f1-a8c9-3d6a247ca21b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.328467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657d0aa-f51a-4f38-a7b8-4ffeff5b33ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.329169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657d0aa-f51a-4f38-a7b8-4ffeff5b33ea admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.329453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657d0aa-f51a-4f38-a7b8-4ffeff5b33ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.329732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657d0aa-f51a-4f38-a7b8-4ffeff5b33ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.330485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657d0aa-f51a-4f38-a7b8-4ffeff5b33ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.332122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657d0aa-f51a-4f38-a7b8-4ffeff5b33ea admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.348279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 940/1888] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:38.349967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:38.360158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f65b1-cdc1-49d8-b74c-f8faf1a53aef admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.365661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.366543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.366849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.367135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] PATH_INFO: `/v3/projects/494e874647ec43b2845ce229b6f571ad/users/eeba0c5ede8d40a5975493d71d22d548/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.369126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=494e874647ec43b2845ce229b6f571ad, user_id=eeba0c5ede8d40a5975493d71d22d548, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.371844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.382688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 949/1889] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:38 2022] PUT /identity/v3/projects/494e874647ec43b2845ce229b6f571ad/users/eeba0c5ede8d40a5975493d71d22d548/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:38.392520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.392839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.393123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.421245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] MFA Rules not processed for user `eeba0c5ede8d40a5975493d71d22d548`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:38.422551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:38.506130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 941/1890] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:38.517286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:38.519043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:38.576065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7e982-e379-4da7-86ce-b37a37ebc802 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:38.579707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4a176-aff4-4a44-89fa-1bb448db074e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:38.580343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4a176-aff4-4a44-89fa-1bb448db074e service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:38.580625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4a176-aff4-4a44-89fa-1bb448db074e service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:38.580897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4a176-aff4-4a44-89fa-1bb448db074e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:38.590144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4a176-aff4-4a44-89fa-1bb448db074e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:38.591588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4a176-aff4-4a44-89fa-1bb448db074e service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:38.601892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 950/1891] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:39.020307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:39.041160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:39.084230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a70f2-731d-4dfc-9dc3-05ed262adcb2 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:39.088772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:39.089636 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:39.089921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:39.090196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] PATH_INFO: `/v3/users/ef351555df3748bbb3d10b7b40396bf3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:39.091405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef351555df3748bbb3d10b7b40396bf3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:39.093334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:39.147266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef351555df3748bbb3d10b7b40396bf3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:39.147874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef351555df3748bbb3d10b7b40396bf3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:39.155105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef351555df3748bbb3d10b7b40396bf3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:39.155739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef351555df3748bbb3d10b7b40396bf3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:39.163960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef351555df3748bbb3d10b7b40396bf3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:39.182293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 942/1892] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:39 2022] DELETE /identity/v3/users/ef351555df3748bbb3d10b7b40396bf3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:39.186777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:39.188489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:39.188782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:39.207432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:39.209151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:39.277792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 951/1893] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:39.290455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:39.292110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:39.374151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b5e82-7332-45a6-aa35-ac9be131802c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:39.385540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4e8e-8b56-4b57-804c-9dd9b71924cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:39.386239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4e8e-8b56-4b57-804c-9dd9b71924cd service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:39.386537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4e8e-8b56-4b57-804c-9dd9b71924cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:39.387000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4e8e-8b56-4b57-804c-9dd9b71924cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:39.424753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4e8e-8b56-4b57-804c-9dd9b71924cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:39.438697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4e8e-8b56-4b57-804c-9dd9b71924cd service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:39.478030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 943/1894] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:39.874020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:39.876687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:39.900074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a008b-f207-4379-994c-f433f7bb9fe2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:39.912416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:39.914483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:39.915754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:39.917329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] PATH_INFO: `/v3/users/acd136a366ca41d5a1e07aa3b88dd190` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:39.927841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd136a366ca41d5a1e07aa3b88dd190)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:39.929938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:40.032244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd136a366ca41d5a1e07aa3b88dd190'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.032969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd136a366ca41d5a1e07aa3b88dd190'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.044981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd136a366ca41d5a1e07aa3b88dd190'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.045876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd136a366ca41d5a1e07aa3b88dd190'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.052971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:40.079356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd136a366ca41d5a1e07aa3b88dd190'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.113905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 952/1895] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:39 2022] DELETE /identity/v3/users/acd136a366ca41d5a1e07aa3b88dd190 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:40.118715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:40.119158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:40.119533 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:40.163662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:40.164627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.237291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:40.309321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 944/1896] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:40.322293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:40.324037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.405804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86812b-7d62-4e17-8e4d-b32d7925fcfd admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:40.412244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113e73a-0b71-445f-bb8b-efa151805c8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:40.421431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113e73a-0b71-445f-bb8b-efa151805c8f service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:40.421922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113e73a-0b71-445f-bb8b-efa151805c8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:40.422307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113e73a-0b71-445f-bb8b-efa151805c8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:40.437294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113e73a-0b71-445f-bb8b-efa151805c8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:40.440789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113e73a-0b71-445f-bb8b-efa151805c8f service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:40.461259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 953/1897] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:40.552559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:40.554141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.576337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e088fb2-0341-4fbe-8032-ed02c557fd60 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:40.579305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:40.580908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:40.581437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:40.581948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] PATH_INFO: `/v3/users/d13ad78d059749ce8a8e85f512138b4e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:40.584289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13ad78d059749ce8a8e85f512138b4e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:40.587305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:40.664773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13ad78d059749ce8a8e85f512138b4e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.665661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13ad78d059749ce8a8e85f512138b4e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.678679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13ad78d059749ce8a8e85f512138b4e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.679829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13ad78d059749ce8a8e85f512138b4e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.689849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13ad78d059749ce8a8e85f512138b4e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.720118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 945/1898] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:40 2022] DELETE /identity/v3/users/d13ad78d059749ce8a8e85f512138b4e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:40.724958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:40.725284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:40.725511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:40.742903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:40.744143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.789680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:40.793485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.844145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 954/1899] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:40.845428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:40.847309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.862770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd8c536-4314-4bea-98e4-c07ee1963b0d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:40.868290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:40.869157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:40.869441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:40.869720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] PATH_INFO: `/v3/projects/26e29d56dda84275b80a36424f352a2f` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:40.870933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e29d56dda84275b80a36424f352a2f)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:40.872908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:40.890515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777f1a4-e65f-4c3d-b0fa-97901f862b81 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:40.895305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:40.896693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:40.896987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:40.897261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] PATH_INFO: `/v3/projects/82eec3d610b44fc18e9e1f2300e5c953` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:40.898743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=82eec3d610b44fc18e9e1f2300e5c953)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:40.900883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:40.921562 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e29d56dda84275b80a36424f352a2f', 'request_id': 'req-cd133cfd-2809-47a0-afac-9d972ae3df6a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.929970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e29d56dda84275b80a36424f352a2f', 'request_id': 'req-cd133cfd-2809-47a0-afac-9d972ae3df6a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.937599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e29d56dda84275b80a36424f352a2f', 'request_id': 'req-cd133cfd-2809-47a0-afac-9d972ae3df6a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.952236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] The token cache is being invalidate because project 26e29d56dda84275b80a36424f352a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:40.952236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.952236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.953308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 955/1900] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:40 2022] DELETE /identity/v3/projects/26e29d56dda84275b80a36424f352a2f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:40.956380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:40.957299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:40.965719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:40.973155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82eec3d610b44fc18e9e1f2300e5c953', 'request_id': 'req-f31c3301-ceda-4fd5-b019-04e0b52780ec'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.991957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82eec3d610b44fc18e9e1f2300e5c953', 'request_id': 'req-f31c3301-ceda-4fd5-b019-04e0b52780ec'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:40.995410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:41.003672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82eec3d610b44fc18e9e1f2300e5c953', 'request_id': 'req-f31c3301-ceda-4fd5-b019-04e0b52780ec'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.018261 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] The token cache is being invalidate because project 82eec3d610b44fc18e9e1f2300e5c953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:41.018673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.019446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.021377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 946/1901] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:40 2022] DELETE /identity/v3/projects/82eec3d610b44fc18e9e1f2300e5c953 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:41.022240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:41.023678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:41.038419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd133cfd-2809-47a0-afac-9d972ae3df6a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:41.092795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:41.093644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:41.093934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:41.094645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:41.096996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:41.098810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31c3301-ceda-4fd5-b019-04e0b52780ec admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:41.106976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:41.108052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:41.108408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:41.108815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] PATH_INFO: `/v3/users/7cc430670b7c4166bc79580fd116ce65` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:41.110946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc430670b7c4166bc79580fd116ce65)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:41.114708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:41.117269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:41.120402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:41.138710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 956/1902] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:41.139792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:41.143738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:41.158150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0477fe-fe2f-46df-9c35-a74313bc9e8f service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:41.164553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:41.165434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:41.165722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:41.165998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] PATH_INFO: `/v3/users/eeba0c5ede8d40a5975493d71d22d548` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:41.167461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeba0c5ede8d40a5975493d71d22d548)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:41.169447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:41.205930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc430670b7c4166bc79580fd116ce65'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.207075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc430670b7c4166bc79580fd116ce65'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.219598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc430670b7c4166bc79580fd116ce65'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.220945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc430670b7c4166bc79580fd116ce65'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.266513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc430670b7c4166bc79580fd116ce65'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.281901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeba0c5ede8d40a5975493d71d22d548'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.283119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeba0c5ede8d40a5975493d71d22d548'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.294180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 947/1903] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:41 2022] DELETE /identity/v3/users/7cc430670b7c4166bc79580fd116ce65 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:41.295071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:41.297277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeba0c5ede8d40a5975493d71d22d548'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.298606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeba0c5ede8d40a5975493d71d22d548'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.311240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:41.313939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeba0c5ede8d40a5975493d71d22d548'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.334704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 957/1904] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:41 2022] DELETE /identity/v3/users/eeba0c5ede8d40a5975493d71d22d548 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:41.436137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80f965-b20d-4a93-966c-51fd3caef607 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:41.453967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:41.602154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:41.603491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:41.604241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:41.604632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] PATH_INFO: `/v3/users/f3d9eb831c1f4d3ca450edd756729b06` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:41.606320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d9eb831c1f4d3ca450edd756729b06)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:41.609101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:41.667402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:41.669597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:41.703117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa188ad-6ce5-4a3e-b1cf-8dfab4089273 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:41.714167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d9eb831c1f4d3ca450edd756729b06'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.715105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d9eb831c1f4d3ca450edd756729b06'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.715423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:41.716307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:41.716600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:41.720291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] PATH_INFO: `/v3/projects/f3dcc6e8c1ea4b5ba1c0b5eb66ee071c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:41.722866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3dcc6e8c1ea4b5ba1c0b5eb66ee071c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:41.728634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:41.732696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d9eb831c1f4d3ca450edd756729b06'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.733498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d9eb831c1f4d3ca450edd756729b06'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.753184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-890caf01-23c7-48c0-a4f8-0c21e4319a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d9eb831c1f4d3ca450edd756729b06'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.783382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 948/1905] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:41 2022] DELETE /identity/v3/users/f3d9eb831c1f4d3ca450edd756729b06 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:41.789148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:41.789573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:41.789959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:41.826964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:41.856207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3dcc6e8c1ea4b5ba1c0b5eb66ee071c', 'request_id': 'req-3c0d7efb-9153-4ee6-b3b0-1082392b439f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.859919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:41.863093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:41.881759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3dcc6e8c1ea4b5ba1c0b5eb66ee071c', 'request_id': 'req-3c0d7efb-9153-4ee6-b3b0-1082392b439f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.890810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3dcc6e8c1ea4b5ba1c0b5eb66ee071c', 'request_id': 'req-3c0d7efb-9153-4ee6-b3b0-1082392b439f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.931124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] The token cache is being invalidate because project f3dcc6e8c1ea4b5ba1c0b5eb66ee071c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:41.931543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.934454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:41.936472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 958/1906] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:41 2022] DELETE /identity/v3/projects/f3dcc6e8c1ea4b5ba1c0b5eb66ee071c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:41.941850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:41.945134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.033579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 949/1907] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:42.047934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.051167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.120808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:42.143638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d7efb-9153-4ee6-b3b0-1082392b439f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.151428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.152394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.152748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.153033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] PATH_INFO: `/v3/users/6e509f8f3e4946aeacb98b7e72fa029e` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.154267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e509f8f3e4946aeacb98b7e72fa029e)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.156422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.191284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5a04e-61b0-495e-b2cf-138505797333 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.202771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.203286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.203825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.204003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.236242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.239070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.239215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e509f8f3e4946aeacb98b7e72fa029e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.240651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e509f8f3e4946aeacb98b7e72fa029e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.264428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 950/1908] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:42.265236 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.269324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.270901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e509f8f3e4946aeacb98b7e72fa029e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.273425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e509f8f3e4946aeacb98b7e72fa029e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.290569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea27bd5-1b7a-4aa5-8d47-6567202a5914 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.296328 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.297205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.306016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e509f8f3e4946aeacb98b7e72fa029e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.308096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.308309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] PATH_INFO: `/v3/projects/c9b230ba239e42eeafe775dcff3387c1` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.309999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b230ba239e42eeafe775dcff3387c1)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.312677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.325238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 959/1909] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:41 2022] DELETE /identity/v3/users/6e509f8f3e4946aeacb98b7e72fa029e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:42.329839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.332595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.386278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b230ba239e42eeafe775dcff3387c1', 'request_id': 'req-2413a531-e269-49db-83bd-ad259dcb7ff2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.390405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:42.394546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b230ba239e42eeafe775dcff3387c1', 'request_id': 'req-2413a531-e269-49db-83bd-ad259dcb7ff2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.404192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b230ba239e42eeafe775dcff3387c1', 'request_id': 'req-2413a531-e269-49db-83bd-ad259dcb7ff2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.422383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] The token cache is being invalidate because project c9b230ba239e42eeafe775dcff3387c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:42.422577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec92a49-571f-4a2c-a5e4-6480325284ca admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.422661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.426238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2413a531-e269-49db-83bd-ad259dcb7ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.433240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 951/1910] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:42 2022] DELETE /identity/v3/projects/c9b230ba239e42eeafe775dcff3387c1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:42.438018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366e47-8d38-48da-8982-0bb0f1db28b2 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.438607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366e47-8d38-48da-8982-0bb0f1db28b2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.439155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366e47-8d38-48da-8982-0bb0f1db28b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.463677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-73366e47-8d38-48da-8982-0bb0f1db28b2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:42.465034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-73366e47-8d38-48da-8982-0bb0f1db28b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.508231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.509193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.509481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.510014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] PATH_INFO: `/v3/projects/494e874647ec43b2845ce229b6f571ad` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.511567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=494e874647ec43b2845ce229b6f571ad)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.513460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.539145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 952/1911] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:42.543082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.543390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.543755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.569744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:42.571042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.577799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494e874647ec43b2845ce229b6f571ad', 'request_id': 'req-260af7a1-a6aa-45c1-869f-85a72d2a48ec'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.585515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494e874647ec43b2845ce229b6f571ad', 'request_id': 'req-260af7a1-a6aa-45c1-869f-85a72d2a48ec'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.592711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494e874647ec43b2845ce229b6f571ad', 'request_id': 'req-260af7a1-a6aa-45c1-869f-85a72d2a48ec'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.606109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] The token cache is being invalidate because project 494e874647ec43b2845ce229b6f571ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:42.606425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.607118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-260af7a1-a6aa-45c1-869f-85a72d2a48ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:42.612330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 960/1912] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:42 2022] DELETE /identity/v3/projects/494e874647ec43b2845ce229b6f571ad => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:42.614864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a89149-7ed0-4308-a856-0a1f11bd0eee None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.615265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a89149-7ed0-4308-a856-0a1f11bd0eee None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.615567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a89149-7ed0-4308-a856-0a1f11bd0eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.651041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-69a89149-7ed0-4308-a856-0a1f11bd0eee None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:42.652564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-69a89149-7ed0-4308-a856-0a1f11bd0eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.670571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:42.674473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 953/1913] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:42.679160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.698219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f111b4c-276a-4847-b7cd-c44c7aed88b6 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.703753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.704912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.705230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.705562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.706227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:42.706542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.708278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.725232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 954/1914] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:42.729583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.742529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199514d-782d-4775-9254-7c7ef3f5f013 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.747405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.748156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.748530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.748876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.749780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.750152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 961/1915] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:42.752611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.758435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.759124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.764240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.784708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:42.785959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:42.790922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 955/1916] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:42.795398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 962/1917] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:18:42.796075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.799364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.812865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bf4d6-6b25-4c22-9ba4-a06dc2d811f3 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.819960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:42.889360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.890980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.891338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.891809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.893191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:42.893678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.895784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.914704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 963/1918] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:42.921963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:42.933088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173432a-b1fa-4115-a374-0ba2d51cba0f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.935445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc36f2-9a38-41b6-b931-4679c5c469a5 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:42.940285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.941754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.942231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.942562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.943403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.945448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.962287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:42.963242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:42.963530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:42.963805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:42.964846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:42.966763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:42.983410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 956/1919] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:42.987168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.016075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 964/1920] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:42 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:43.017728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af125d-b31a-4b53-a908-63cdd9cbc4b6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.020384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.104135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c2fd7-72c0-4610-a9f5-5ea90518b9ab admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.105076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.105915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.106480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.107076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.107971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.109437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.110106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.110336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.110464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.110927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.111522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.113442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.127416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 957/1921] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:43.128665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.131120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.144670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 965/1922] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:43.145725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.146270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e8a4-2a91-4fd7-9084-40587a967fbf admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.147682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.152545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.153723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.154213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.154853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] PATH_INFO: `/v3/projects/b63d1b04401349d2bb1e2bca81d12850` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.156289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63d1b04401349d2bb1e2bca81d12850)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.158673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.163076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4906b67-8eb5-4f9f-91dc-a56cff9e46af admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.167841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.168707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.168993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.169271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] PATH_INFO: `/v3/projects/791eab758a8b4db98e721ce1fbcd46d1` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.170447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=791eab758a8b4db98e721ce1fbcd46d1)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.172545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.213047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:43.218968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63d1b04401349d2bb1e2bca81d12850', 'request_id': 'req-fde5294b-9c26-4429-9199-803d6f6a4abb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.229044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63d1b04401349d2bb1e2bca81d12850', 'request_id': 'req-fde5294b-9c26-4429-9199-803d6f6a4abb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.230977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '791eab758a8b4db98e721ce1fbcd46d1', 'request_id': 'req-0c532412-8fe8-4982-9df6-63cacec3de89'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.237931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63d1b04401349d2bb1e2bca81d12850', 'request_id': 'req-fde5294b-9c26-4429-9199-803d6f6a4abb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.241177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '791eab758a8b4db98e721ce1fbcd46d1', 'request_id': 'req-0c532412-8fe8-4982-9df6-63cacec3de89'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.249411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '791eab758a8b4db98e721ce1fbcd46d1', 'request_id': 'req-0c532412-8fe8-4982-9df6-63cacec3de89'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.250802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:43.251417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] The token cache is being invalidate because project b63d1b04401349d2bb1e2bca81d12850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:43.251786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.252636 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.254475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 958/1923] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:43 2022] DELETE /identity/v3/projects/b63d1b04401349d2bb1e2bca81d12850 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:43.255408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.256815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.266959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] The token cache is being invalidate because project 791eab758a8b4db98e721ce1fbcd46d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:43.268269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.269286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:43.272561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 966/1924] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:43 2022] DELETE /identity/v3/projects/791eab758a8b4db98e721ce1fbcd46d1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:43.284918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.295462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.347494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5294b-9c26-4429-9199-803d6f6a4abb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.380705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532412-8fe8-4982-9df6-63cacec3de89 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.387294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a7a17-3c6f-418c-900e-3b344629ce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.388042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7a17-3c6f-418c-900e-3b344629ce47 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.388327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7a17-3c6f-418c-900e-3b344629ce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.388604 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7a17-3c6f-418c-900e-3b344629ce47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.389522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a7a17-3c6f-418c-900e-3b344629ce47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.391368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a7a17-3c6f-418c-900e-3b344629ce47 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.405097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 967/1925] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:43.415564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98420c27-584f-4b0a-8188-532240e5383f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.424943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98420c27-584f-4b0a-8188-532240e5383f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.432091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.433701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98420c27-584f-4b0a-8188-532240e5383f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.435805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.436135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.436473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.437516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.440121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.455511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 959/1926] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:43.459697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.482776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-98420c27-584f-4b0a-8188-532240e5383f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:43.482776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-98420c27-584f-4b0a-8188-532240e5383f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.484019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41a052-9b74-45d2-8848-05c403291823 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.490729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 968/1927] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:43.491260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98420c27-584f-4b0a-8188-532240e5383f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.491638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71e12d-4aa9-4d13-92d7-c3d6e2685083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.492518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71e12d-4aa9-4d13-92d7-c3d6e2685083 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.492786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71e12d-4aa9-4d13-92d7-c3d6e2685083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.493069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71e12d-4aa9-4d13-92d7-c3d6e2685083 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.493809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71e12d-4aa9-4d13-92d7-c3d6e2685083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.495662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71e12d-4aa9-4d13-92d7-c3d6e2685083 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.497291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-98420c27-584f-4b0a-8188-532240e5383f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.508552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 960/1928] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:43.510029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.510458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.510927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.520585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98420c27-584f-4b0a-8188-532240e5383f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.527277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.528250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.528515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.528784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] PATH_INFO: `/v3/projects/1d1b11c8f48949b6aaa44929afafd3d0/users/7dc0930763a64c26ac235c4c64363890/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.529791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:43.531036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.543247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 961/1929] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:43.544401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.549391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d1b11c8f48949b6aaa44929afafd3d0, user_id=7dc0930763a64c26ac235c4c64363890, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.553355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.555270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316bad1-bd04-4058-ac23-0aa39fdf40bd None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.560795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955f4562-9ccd-4a9a-b97f-95965f58ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.561677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f4562-9ccd-4a9a-b97f-95965f58ceb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.561963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f4562-9ccd-4a9a-b97f-95965f58ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.562346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f4562-9ccd-4a9a-b97f-95965f58ceb4 admin admin] PATH_INFO: `/v3/projects/b79c7eab7edf4beda1efb340d2869a31/users/d73c3692c8864ce9a6f3c7f3d2524c16/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.569667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 969/1930] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:43 2022] PUT /identity/v3/projects/1d1b11c8f48949b6aaa44929afafd3d0/users/7dc0930763a64c26ac235c4c64363890/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:43.575966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.599625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f4562-9ccd-4a9a-b97f-95965f58ceb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79c7eab7edf4beda1efb340d2869a31, user_id=d73c3692c8864ce9a6f3c7f3d2524c16, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.603535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955f4562-9ccd-4a9a-b97f-95965f58ceb4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.622673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 962/1931] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:43 2022] PUT /identity/v3/projects/b79c7eab7edf4beda1efb340d2869a31/users/d73c3692c8864ce9a6f3c7f3d2524c16/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:43.634040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfba615-4541-4f08-970c-3a25538670b4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.635207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfba615-4541-4f08-970c-3a25538670b4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.636336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfba615-4541-4f08-970c-3a25538670b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.698518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354fbd3-575b-4a01-a162-52129c9e8ea7 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.711073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bdfba615-4541-4f08-970c-3a25538670b4 None None] MFA Rules not processed for user `7dc0930763a64c26ac235c4c64363890`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:43.714459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bdfba615-4541-4f08-970c-3a25538670b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.767558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:43.827242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.827925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.828226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.828476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.829224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:43.829538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.831205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.856768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:43.860188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 963/1932] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:43.867263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.867670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.868058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.879373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 970/1933] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:43.879373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:43.892198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511cd4d-8c95-481f-99ee-b13e7b15c1ec admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:43.901229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:43.906445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:43.907232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:43.908181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:43.909335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:43.912643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] MFA Rules not processed for user `d73c3692c8864ce9a6f3c7f3d2524c16`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:43.913881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:43.914175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:43.950407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 971/1934] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:43.951785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.009921 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 964/1935] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:44.014241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.028185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8140e-4255-4e63-a459-7d4bae652b11 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.073433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2536803f-3edc-4866-a94a-eb9224cc7f81 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.079226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.080610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.080895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.081168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.082100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.086035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.112397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.113620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.113953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.114970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.119142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.124452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 972/1936] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:43 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:44.124452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.124653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.161971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 965/1937] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:44.164885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.188244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384217a-22c8-4c7a-b0e7-39606c51972f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.242189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bbccd-bcb6-449e-a3e3-1e5379f704dd admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.249506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.250611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.250963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.251248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.251982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.254103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.266090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:44.291597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.292487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.292796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.293165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.293306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 973/1938] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:44.294131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.294240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.296813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.329643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 966/1939] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:44.329643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.342089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a221ae-deef-4fb2-a1d3-c7fe83f35cb2 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.412559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:44.431100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c2694-c3d4-4ac5-a3b3-e2e8b9ed325c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.437037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.437496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.437777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.438131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.438263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.438526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.438998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.439101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.439524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.440403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.440927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.443603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.454867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 974/1940] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:44.455820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.466444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 967/1941] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:44.467163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04a12-6135-438a-bf43-467e260ac287 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.467800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.474981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.475722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.476067 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.476457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.477266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.479058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.479822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9036781-f8b5-4aee-a744-280939346f3f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.484823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.485575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.485856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.486147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.486989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.488571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.490809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 975/1942] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:44.494708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.500557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 968/1943] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:44.501557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.506985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b008a10-74e4-4a4b-96d3-98c16fe2c76e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.511470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4da27-356c-4a76-a58b-c35eb278507a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.511616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.512296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.512579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.513142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.513721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.515716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.516259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.517015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.517191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.517474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.518193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.519840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.526308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 976/1944] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:44.527146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.532459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 969/1945] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:44.533650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.537899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fcd7e-bc67-499a-993e-41871e8395cb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.542654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.543555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.543834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.544108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] PATH_INFO: `/v3/projects/aafa292a73574731832ac3a82f1b9e08/users/bd26b88551d54d1a8ff3d546fe4a0a2f/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.544540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01450be-d812-42f4-b6e1-d974a4e70952 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.549269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.550133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.550416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.550689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] PATH_INFO: `/v3/projects/0958d610c9d3405183b2d30452e43dba/users/f1b697a54294423e9126e7c822c65f1b/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.559946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aafa292a73574731832ac3a82f1b9e08, user_id=bd26b88551d54d1a8ff3d546fe4a0a2f, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.562767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.565989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0958d610c9d3405183b2d30452e43dba, user_id=f1b697a54294423e9126e7c822c65f1b, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.568712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.576826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 977/1946] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:44 2022] PUT /identity/v3/projects/aafa292a73574731832ac3a82f1b9e08/users/bd26b88551d54d1a8ff3d546fe4a0a2f/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:44.577971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.582100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 970/1947] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:44 2022] PUT /identity/v3/projects/0958d610c9d3405183b2d30452e43dba/users/f1b697a54294423e9126e7c822c65f1b/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:44.585994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.647366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaad4d-4279-4ff0-aec7-110506311f19 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.648708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dc5d7-9429-4987-8e01-ba8ca0431dee admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.652205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.653090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.653375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.653518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74eeb4a-637a-4fed-bb51-d28fb277e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.653638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] PATH_INFO: `/v3/projects/d7825374f809425688a5767b8c79e828/users/5c76de7a9cd541eba48819222b5cfc07/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.654175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eeb4a-637a-4fed-bb51-d28fb277e91c admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.654460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eeb4a-637a-4fed-bb51-d28fb277e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.654736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eeb4a-637a-4fed-bb51-d28fb277e91c admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.655515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74eeb4a-637a-4fed-bb51-d28fb277e91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.657122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74eeb4a-637a-4fed-bb51-d28fb277e91c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.667521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 971/1948] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:44.668739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2574-16da-4913-bc26-3e4e64e07056 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.669211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2574-16da-4913-bc26-3e4e64e07056 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.669496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d2574-16da-4913-bc26-3e4e64e07056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.672127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7825374f809425688a5767b8c79e828, user_id=5c76de7a9cd541eba48819222b5cfc07, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.674978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.690227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:44.694357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 978/1949] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:44 2022] PUT /identity/v3/projects/d7825374f809425688a5767b8c79e828/users/5c76de7a9cd541eba48819222b5cfc07/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:44.694357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.699721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-854d2574-16da-4913-bc26-3e4e64e07056 None None] MFA Rules not processed for user `bd26b88551d54d1a8ff3d546fe4a0a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:44.701863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-854d2574-16da-4913-bc26-3e4e64e07056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:44.760960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dd7e9-522f-40ca-8be4-5cfd5f5d61a1 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.765716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.766599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.766913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.767192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] PATH_INFO: `/v3/projects/0958d610c9d3405183b2d30452e43dba/users/f1b697a54294423e9126e7c822c65f1b/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.769664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0958d610c9d3405183b2d30452e43dba, user_id=f1b697a54294423e9126e7c822c65f1b, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.773274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.773407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 972/1950] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:44.776201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51e210-f250-4280-9c70-f791e2a601a7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.776490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51e210-f250-4280-9c70-f791e2a601a7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.776776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51e210-f250-4280-9c70-f791e2a601a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.785620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 979/1951] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:44 2022] PUT /identity/v3/projects/0958d610c9d3405183b2d30452e43dba/users/f1b697a54294423e9126e7c822c65f1b/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:44.791034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.800536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bb51e210-f250-4280-9c70-f791e2a601a7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:44.801802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bb51e210-f250-4280-9c70-f791e2a601a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:44.807248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:44.854820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe0859-3bdd-4adc-9d91-f07f08973a83 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.859702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.860411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.860698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.861009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.861703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:44.863658 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:44.868555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 973/1952] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:44.869954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.870252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.870526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:44.894936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 980/1953] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:44.896058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:44.899443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:44.901587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] MFA Rules not processed for user `f1b697a54294423e9126e7c822c65f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:44.903110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:44.975049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dff0ac-3d67-4be3-9986-3f8bc04ad224 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:44.981641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:44.981641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:44.981641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:44.981641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.025315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 974/1954] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:45.026515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.056752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.058337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.071232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 981/1955] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:45.072195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.073912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:45.091021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:45.143971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e23a6-a700-4728-87cc-8c1618ad7eed None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.149546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.150372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.151418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.152066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.152912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.155185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.186648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 975/1956] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:45 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:45.191609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.215347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88981fe2-69b1-4eb5-87ef-094704af9a31 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.216239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52258a1e-9950-490a-ad5a-9aeee2187e04 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.223916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.224340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.224977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.225090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.225298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.225416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.225647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.226472 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.227296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.229070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.232554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:45.239119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.241807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.246943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 976/1957] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:45.247885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.259339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 982/1958] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:45.268787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83050f-8ffd-4fa1-97a5-536f7d6c038c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.284569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.289031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.291317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.291628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] PATH_INFO: `/v3/projects/cba91f3ab34c4e4eacea21e385715062/users/673fdd8fe1ad4605bc7d16b2d64f5185/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.343838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba91f3ab34c4e4eacea21e385715062, user_id=673fdd8fe1ad4605bc7d16b2d64f5185, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.350022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.374631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 977/1959] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:45 2022] PUT /identity/v3/projects/cba91f3ab34c4e4eacea21e385715062/users/673fdd8fe1ad4605bc7d16b2d64f5185/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:45.382120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.393630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd28a2b-60ca-46ff-8ad8-8f2dff5a1606 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.395937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.511774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1401e-4ebf-4800-bfd5-1382ecb8f31b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.515936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.516673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.517030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.517411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.555263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.556134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.556540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.556925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.558192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.560717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.591002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 983/1960] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:45.601352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.618614 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e992459-9331-4b63-a0ca-f2e676ee8663 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.624923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dc7bf-80f6-4bf4-aef0-55f570017974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:45.625841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dc7bf-80f6-4bf4-aef0-55f570017974 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.626126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dc7bf-80f6-4bf4-aef0-55f570017974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.626407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dc7bf-80f6-4bf4-aef0-55f570017974 admin admin] PATH_INFO: `/v3/projects/cba91f3ab34c4e4eacea21e385715062/users/673fdd8fe1ad4605bc7d16b2d64f5185/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.629245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dc7bf-80f6-4bf4-aef0-55f570017974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba91f3ab34c4e4eacea21e385715062, user_id=673fdd8fe1ad4605bc7d16b2d64f5185, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.633028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dc7bf-80f6-4bf4-aef0-55f570017974 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.643320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:45.645956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:45.654882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 984/1961] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:45 2022] PUT /identity/v3/projects/cba91f3ab34c4e4eacea21e385715062/users/673fdd8fe1ad4605bc7d16b2d64f5185/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:45.657959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:45.659178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:45.659629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:45.679577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 978/1962] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:45.696834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] MFA Rules not processed for user `673fdd8fe1ad4605bc7d16b2d64f5185`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:45.699620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:45.871583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 985/1963] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:45.880572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:45.908043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b6b82-f2cb-4853-844b-32dfad845ef8 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:45.976931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:46.000338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed856a09-104b-45f1-a67f-543cf485652c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:46.001112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed856a09-104b-45f1-a67f-543cf485652c service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:46.001488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed856a09-104b-45f1-a67f-543cf485652c service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:46.001829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed856a09-104b-45f1-a67f-543cf485652c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:46.014225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed856a09-104b-45f1-a67f-543cf485652c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:46.016508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed856a09-104b-45f1-a67f-543cf485652c service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:46.031545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 979/1964] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:46.073811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:46.075263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:46.088348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54a7bc7-db87-468c-8c18-36dd791fa2a0 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:46.092164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:46.092917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:46.093305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:46.093682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:46.195044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:46.197143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:46.202896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:46.209824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 986/1965] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:46.776776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:46.777329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:46.777423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:46.806473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] MFA Rules not processed for user `5c76de7a9cd541eba48819222b5cfc07`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:46.808427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:46.909425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 980/1966] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:46.914786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:47.057837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcdfe9-aad0-400b-92b2-bb7cc13539fa service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:47.067906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:47.068508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:47.068803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:47.069076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:47.069799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:47.071857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:47.127583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:47.151236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 987/1967] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:47.171339 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba72f6-1624-4e5b-8097-3b2b8d554800 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:47.288834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:47.289856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:47.290441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:47.290912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:47.291634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:47.293662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:47.330098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 981/1968] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:47 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:47.331733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:47.346619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc78378-ce2e-49df-9cbe-af2b93264c69 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:47.363675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:47.364928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:47.365210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:47.365537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:47.366645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:47.369850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:47.383503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 988/1969] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:47.387737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:47.398605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d636a39-006a-407a-b5de-1574b9e05302 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:47.404597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:47.405594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:47.405920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:47.406200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] PATH_INFO: `/v3/projects/831f4787578246b2a26996696ad59617/users/562ec6e2a9a34276ab4ade021eb0d33f/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:47.426057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=831f4787578246b2a26996696ad59617, user_id=562ec6e2a9a34276ab4ade021eb0d33f, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:47.432389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:47.450220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 982/1970] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:47 2022] PUT /identity/v3/projects/831f4787578246b2a26996696ad59617/users/562ec6e2a9a34276ab4ade021eb0d33f/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:47.454366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:47.466055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23203bb6-af4e-4169-b7ce-5a0484b8dbd2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:47.545533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7b8bb-a438-4611-9ea1-007b3ade380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:47.546572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b8bb-a438-4611-9ea1-007b3ade380c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:47.547032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b8bb-a438-4611-9ea1-007b3ade380c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:47.548126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7b8bb-a438-4611-9ea1-007b3ade380c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:47.549103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7b8bb-a438-4611-9ea1-007b3ade380c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:47.551038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7b8bb-a438-4611-9ea1-007b3ade380c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:47.567289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 989/1971] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:18:47.571914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:47.589083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295250ea-fa32-4f8d-9398-897612b4f9eb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:47.600463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:47.601987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:47.602538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:47.602996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] PATH_INFO: `/v3/projects/831f4787578246b2a26996696ad59617/users/562ec6e2a9a34276ab4ade021eb0d33f/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:47.606577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=831f4787578246b2a26996696ad59617, user_id=562ec6e2a9a34276ab4ade021eb0d33f, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:47.610122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:47.629378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 983/1972] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:47 2022] PUT /identity/v3/projects/831f4787578246b2a26996696ad59617/users/562ec6e2a9a34276ab4ade021eb0d33f/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:48.365602 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:48.366175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:48.366175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:48.402939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] MFA Rules not processed for user `562ec6e2a9a34276ab4ade021eb0d33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:48.404437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:48.491659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:48.509182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 990/1973] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:48.518052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:48.550643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:48.596430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778821d-25b1-4f76-a6a4-ceb9bca33d9a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:48.600299 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:48.600954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:48.601241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:48.601914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:48.665014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:48.666630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:48.675890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 984/1974] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:51.212792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:51.222858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22178bb9-71e6-466c-9647-1d7a6a10c8fa None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:51.229609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0c81b-ca44-4cc0-8d48-590f9dccbc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:51.230342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0c81b-ca44-4cc0-8d48-590f9dccbc0c service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:51.230692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0c81b-ca44-4cc0-8d48-590f9dccbc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:51.231082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0c81b-ca44-4cc0-8d48-590f9dccbc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:51.243274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0c81b-ca44-4cc0-8d48-590f9dccbc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:51.244837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0c81b-ca44-4cc0-8d48-590f9dccbc0c service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:51.254691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 991/1975] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:51.595340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:51.693962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92aa63-bef2-4368-8220-dfdea5bac6a4 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:51.703630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:51.704597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:51.704955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:51.705300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] PATH_INFO: `/v3/users/7dc0930763a64c26ac235c4c64363890` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:51.707068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc0930763a64c26ac235c4c64363890)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:51.709675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:51.770878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc0930763a64c26ac235c4c64363890'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:51.771689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc0930763a64c26ac235c4c64363890'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:51.779377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc0930763a64c26ac235c4c64363890'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:51.781797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc0930763a64c26ac235c4c64363890'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:51.789364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc0930763a64c26ac235c4c64363890'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:51.805412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:51.821114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 985/1976] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:51 2022] DELETE /identity/v3/users/7dc0930763a64c26ac235c4c64363890 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:51.825201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:51.825497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:51.825852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:51.844347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:51.845557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:51.924716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 992/1977] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:51.939145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:51.942010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:52.018232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9f2f-5fd4-4dca-91d4-0ed60418913b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:52.021983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:52.022628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:52.022939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:52.023217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:52.032772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:52.034462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:52.044733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 986/1978] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:52.325259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:52.326847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:52.338848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203c0eb-1517-4c01-b6ed-872eb5f76c6f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:52.343519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:52.344446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:52.344717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:52.344995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] PATH_INFO: `/v3/projects/1d1b11c8f48949b6aaa44929afafd3d0` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:52.346443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d1b11c8f48949b6aaa44929afafd3d0)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:52.348412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:52.394661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d1b11c8f48949b6aaa44929afafd3d0', 'request_id': 'req-4b708ab1-053b-40d1-a1d9-787b45e204c5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.401761 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d1b11c8f48949b6aaa44929afafd3d0', 'request_id': 'req-4b708ab1-053b-40d1-a1d9-787b45e204c5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.407181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d1b11c8f48949b6aaa44929afafd3d0', 'request_id': 'req-4b708ab1-053b-40d1-a1d9-787b45e204c5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.417946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] The token cache is being invalidate because project 1d1b11c8f48949b6aaa44929afafd3d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:52.418369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.419098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.420657 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 993/1979] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:52 2022] DELETE /identity/v3/projects/1d1b11c8f48949b6aaa44929afafd3d0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:52.424793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:52.434945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ce5db-fb2b-47bd-aed6-cf03e7df4fba service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:52.492434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:52.493302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:52.493588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:52.493860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] PATH_INFO: `/v3/users/bd26b88551d54d1a8ff3d546fe4a0a2f` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:52.495066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd26b88551d54d1a8ff3d546fe4a0a2f)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:52.496983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:52.542296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd26b88551d54d1a8ff3d546fe4a0a2f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.542883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd26b88551d54d1a8ff3d546fe4a0a2f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.549720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd26b88551d54d1a8ff3d546fe4a0a2f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.550335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd26b88551d54d1a8ff3d546fe4a0a2f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.557170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd26b88551d54d1a8ff3d546fe4a0a2f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:52.574811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 987/1980] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:52 2022] DELETE /identity/v3/users/bd26b88551d54d1a8ff3d546fe4a0a2f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:52.845713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:52.848083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:52.862810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b708ab1-053b-40d1-a1d9-787b45e204c5 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:52.929739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:52.930617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:52.930927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:52.931205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] PATH_INFO: `/v3/projects/aafa292a73574731832ac3a82f1b9e08` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:52.932810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aafa292a73574731832ac3a82f1b9e08)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:52.935517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:52.945840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:53.000727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aafa292a73574731832ac3a82f1b9e08', 'request_id': 'req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.020652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aafa292a73574731832ac3a82f1b9e08', 'request_id': 'req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.031483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aafa292a73574731832ac3a82f1b9e08', 'request_id': 'req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.043187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] The token cache is being invalidate because project aafa292a73574731832ac3a82f1b9e08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:53.043474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.044114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.045672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 994/1981] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:52 2022] DELETE /identity/v3/projects/aafa292a73574731832ac3a82f1b9e08 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:53.049689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:53.051241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:53.065239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbde16-ae06-48c9-ad5b-876461fd74d5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:53.081505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:53.130561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:53.131457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.131741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.132016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] PATH_INFO: `/v3/users/673fdd8fe1ad4605bc7d16b2d64f5185` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.133223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=673fdd8fe1ad4605bc7d16b2d64f5185)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:53.135190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:53.187189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673fdd8fe1ad4605bc7d16b2d64f5185'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.188052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673fdd8fe1ad4605bc7d16b2d64f5185'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.196745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673fdd8fe1ad4605bc7d16b2d64f5185'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.197433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673fdd8fe1ad4605bc7d16b2d64f5185'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.205594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c165cee4-aca0-4726-9e27-351426f0f012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673fdd8fe1ad4605bc7d16b2d64f5185'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.223884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 988/1982] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:53 2022] DELETE /identity/v3/users/673fdd8fe1ad4605bc7d16b2d64f5185 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:53.496241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:53.497652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:53.511094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a584c1-9e42-453d-90ed-a7cfa8bdc08d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:53.568939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:53.569798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.570076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.570356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] PATH_INFO: `/v3/projects/cba91f3ab34c4e4eacea21e385715062` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.571526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba91f3ab34c4e4eacea21e385715062)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:53.573415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:53.618534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba91f3ab34c4e4eacea21e385715062', 'request_id': 'req-9e3755de-4708-4a0d-9ced-78b24a2bca3d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.625523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba91f3ab34c4e4eacea21e385715062', 'request_id': 'req-9e3755de-4708-4a0d-9ced-78b24a2bca3d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.631120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba91f3ab34c4e4eacea21e385715062', 'request_id': 'req-9e3755de-4708-4a0d-9ced-78b24a2bca3d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.641617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] The token cache is being invalidate because project cba91f3ab34c4e4eacea21e385715062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:53.641902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.642519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e3755de-4708-4a0d-9ced-78b24a2bca3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:53.644056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 995/1983] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:53 2022] DELETE /identity/v3/projects/cba91f3ab34c4e4eacea21e385715062 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:53.649873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.650163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.650438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.668552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:53.669738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:53.731083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 989/1984] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:53.736799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.737092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.737371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.755432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:53.756626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:53.764462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 996/1985] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:53.768367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:53.778323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c235d4-b5e2-449b-8764-c4162aac1539 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:53.782790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:53.783453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.783740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.784015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.784736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:18:53.785040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:53.787288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:53.798608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 990/1986] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:18:53.802718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:53.811820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa17933-c7dd-4848-8985-edbeffb5b2b6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:53.816451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:53.817084 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.817363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.817634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.818324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:53.820222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:53.839134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:53.849269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 997/1987] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:53 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:53.853185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:53.863094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500337f-0407-4ee4-8f9f-e11bb82d62d0 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:53.882676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:53.925367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:53.926033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.926336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.926658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.927377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:53.929292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:53.956268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 991/1988] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:53 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:53.960815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:53.971353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a80012-2499-48b7-ad49-eadf6267566a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:53.976685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:53.977379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:53.977677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:53.977954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:53.978677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:53.982653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:53.991191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 998/1989] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:53.994877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.006574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf69a55-e50f-45e5-a42d-96774b4b2375 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.011593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95103b46-3ff8-4e46-beff-23881cf896e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.012253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95103b46-3ff8-4e46-beff-23881cf896e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.012536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95103b46-3ff8-4e46-beff-23881cf896e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.012818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95103b46-3ff8-4e46-beff-23881cf896e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.013568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95103b46-3ff8-4e46-beff-23881cf896e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.015200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95103b46-3ff8-4e46-beff-23881cf896e6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.029156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 992/1990] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:54.030711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.048220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d73776-cc89-4290-ab6d-e7091653c6f8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.053373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.054269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.054620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.054994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] PATH_INFO: `/v3/projects/501b8e05b7a942df9c7a748d8be3c965/users/5cc7a6d51eab4706827d125415c63a6b/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.072507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501b8e05b7a942df9c7a748d8be3c965, user_id=5cc7a6d51eab4706827d125415c63a6b, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.075216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.089724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 999/1991] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:54 2022] PUT /identity/v3/projects/501b8e05b7a942df9c7a748d8be3c965/users/5cc7a6d51eab4706827d125415c63a6b/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:54.101315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.101779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.101885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.129176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] MFA Rules not processed for user `5cc7a6d51eab4706827d125415c63a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:54.130823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:54.193504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 993/1992] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:54.198391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.261392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec7698-f689-4591-b405-98a7ed72fe82 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.266142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.266803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.267115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.267394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.268097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.269982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.296926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1000/1993] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:18:54 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:18:54.301077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.311807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21270f-239c-4243-b3aa-9792b5fbfb8c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.371946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.372695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.372984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.373265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.373950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.375922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.400617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 994/1994] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:18:54 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:18:54.401844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.412483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a602f-f0b6-48a0-8558-300cc2f20177 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.417204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.417874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.418159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.418434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.419388 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.420812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.440429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1001/1995] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:54.447502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.455370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540d0ad-99f1-4d18-8d25-e848a562fdbb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.460032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.460907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.461194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.461475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] PATH_INFO: `/v3/projects/ec8bf6f1c8de4be391d75d4a8d00c6bd/users/a7fe1c957bde4ec6b3360d91a98910b2/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.476200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8bf6f1c8de4be391d75d4a8d00c6bd, user_id=a7fe1c957bde4ec6b3360d91a98910b2, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.478889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.481845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:54.493586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 995/1996] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:54 2022] PUT /identity/v3/projects/ec8bf6f1c8de4be391d75d4a8d00c6bd/users/a7fe1c957bde4ec6b3360d91a98910b2/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:54.497431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.507969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a0ac9-40b8-46f3-a64b-81ffa121c2fd admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.562178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b735dad-2f6d-44da-9f54-7c42f868ad00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.562845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735dad-2f6d-44da-9f54-7c42f868ad00 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.563174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735dad-2f6d-44da-9f54-7c42f868ad00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.563452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735dad-2f6d-44da-9f54-7c42f868ad00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.564175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b735dad-2f6d-44da-9f54-7c42f868ad00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.565767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b735dad-2f6d-44da-9f54-7c42f868ad00 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.575911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1002/1997] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:18:54.580245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.590365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972bbff-0be4-43db-9b91-49fcce138d0b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.594908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.595775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.596313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.596583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] PATH_INFO: `/v3/projects/ec8bf6f1c8de4be391d75d4a8d00c6bd/users/a7fe1c957bde4ec6b3360d91a98910b2/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.598518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8bf6f1c8de4be391d75d4a8d00c6bd, user_id=a7fe1c957bde4ec6b3360d91a98910b2, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.601159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.610884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 996/1998] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:18:54 2022] PUT /identity/v3/projects/ec8bf6f1c8de4be391d75d4a8d00c6bd/users/a7fe1c957bde4ec6b3360d91a98910b2/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:18:54.621799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87480614-3005-4ec3-af4c-002313aea88a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.622087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87480614-3005-4ec3-af4c-002313aea88a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.622367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87480614-3005-4ec3-af4c-002313aea88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.648752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-87480614-3005-4ec3-af4c-002313aea88a None None] MFA Rules not processed for user `a7fe1c957bde4ec6b3360d91a98910b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:54.649894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-87480614-3005-4ec3-af4c-002313aea88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:54.669444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:54.717763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1003/1999] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:54.729023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:54.730691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:54.786991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba8669-23e0-46ca-be0e-4315e097cd2c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:54.791023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b777a7-d9e2-4700-97a0-f4bc3cbefa7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:54.791681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b777a7-d9e2-4700-97a0-f4bc3cbefa7a service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:54.791964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b777a7-d9e2-4700-97a0-f4bc3cbefa7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:54.792242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b777a7-d9e2-4700-97a0-f4bc3cbefa7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:54.847129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b777a7-d9e2-4700-97a0-f4bc3cbefa7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:54.849061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b777a7-d9e2-4700-97a0-f4bc3cbefa7a service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:54.857758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 997/2000] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:58.281275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87480614-3005-4ec3-af4c-002313aea88a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:58.282844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-87480614-3005-4ec3-af4c-002313aea88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:58.368347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87480614-3005-4ec3-af4c-002313aea88a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:58.372942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:58.373826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:58.374109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:58.374383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] PATH_INFO: `/v3/users/e56fdbebcd3741ae98874e51a8cc9104` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:58.375630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56fdbebcd3741ae98874e51a8cc9104)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:58.377570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:58.424597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56fdbebcd3741ae98874e51a8cc9104'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:58.425241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56fdbebcd3741ae98874e51a8cc9104'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:58.433347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56fdbebcd3741ae98874e51a8cc9104'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:58.434420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56fdbebcd3741ae98874e51a8cc9104'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:58.447375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56fdbebcd3741ae98874e51a8cc9104'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:58.461600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1004/2001] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:58 2022] DELETE /identity/v3/users/e56fdbebcd3741ae98874e51a8cc9104 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:58.466119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:58.466656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:58.466656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:58.484271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:58.485518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:58.545473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 998/2002] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:58.555639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:58.557076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:58.599405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:58.616785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244fc93-b469-40bf-a4f5-c1dd4859b8b8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:58.620420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:58.621057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:58.621338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:58.621609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:58.630624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:58.632161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:58.643265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1005/2003] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:18:58.919978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:58.921855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:58.942309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64cd84-1057-48f9-93ba-2c5338bd12d9 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:58.947295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:58.948357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:58.948808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:58.949246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] PATH_INFO: `/v3/projects/5729496a93514450b7b7cda4ad736ba6` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:58.950848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5729496a93514450b7b7cda4ad736ba6)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:58.954227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:58.997638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:59.012614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5729496a93514450b7b7cda4ad736ba6', 'request_id': 'req-5f834597-de5c-4440-aa44-b9e75f95c91f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.020072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5729496a93514450b7b7cda4ad736ba6', 'request_id': 'req-5f834597-de5c-4440-aa44-b9e75f95c91f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.034505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5729496a93514450b7b7cda4ad736ba6', 'request_id': 'req-5f834597-de5c-4440-aa44-b9e75f95c91f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.051754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] The token cache is being invalidate because project 5729496a93514450b7b7cda4ad736ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:59.052333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.053394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.055389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 999/2004] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:58 2022] DELETE /identity/v3/projects/5729496a93514450b7b7cda4ad736ba6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:59.059541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:59.069893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fface9-24fb-4771-8c1e-46343d72b7fe service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:59.160777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:59.163590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:59.163881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:59.164191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] PATH_INFO: `/v3/users/2ec748790a894fa3b9751942c3c6011d` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:59.165735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec748790a894fa3b9751942c3c6011d)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:59.168843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:59.243838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec748790a894fa3b9751942c3c6011d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.244890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec748790a894fa3b9751942c3c6011d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.253490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec748790a894fa3b9751942c3c6011d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.254507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec748790a894fa3b9751942c3c6011d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.266494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec748790a894fa3b9751942c3c6011d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.283875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1006/2005] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:59 2022] DELETE /identity/v3/users/2ec748790a894fa3b9751942c3c6011d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:59.383027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:59.384990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.399445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f834597-de5c-4440-aa44-b9e75f95c91f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:59.447359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:59.449574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.519311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19122b74-170e-4286-84c4-a7c41858c4ea admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:59.523772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:59.524817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:59.524817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:59.525035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:59.526995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.527374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:59.530761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:59.531241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:59.531794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] PATH_INFO: `/v3/users/d73c3692c8864ce9a6f3c7f3d2524c16` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:59.533958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73c3692c8864ce9a6f3c7f3d2524c16)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:59.536542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:59.544851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:59.615829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73c3692c8864ce9a6f3c7f3d2524c16'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.616689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73c3692c8864ce9a6f3c7f3d2524c16'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.627108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:59.627404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73c3692c8864ce9a6f3c7f3d2524c16'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.628387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73c3692c8864ce9a6f3c7f3d2524c16'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.629495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:59.642801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c2cdec7d-7361-4dd9-820d-041706c43b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73c3692c8864ce9a6f3c7f3d2524c16'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.648825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1007/2006] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:18:59.650142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:59.661438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.667375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1000/2007] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:18:59 2022] DELETE /identity/v3/users/d73c3692c8864ce9a6f3c7f3d2524c16 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:59.671889 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:59.672588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:59.673031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:59.683034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cac8c-e334-4452-8f1e-c07caa61590a service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:59.699738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:59.701338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.749135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:18:59.765844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:18:59.767521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:59.768474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:59.768831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] PATH_INFO: `/v3/projects/2d09c41d6ea14efaa98f4a964ccc928b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:59.771532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d09c41d6ea14efaa98f4a964ccc928b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:18:59.775154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:18:59.790712 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1001/2008] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:18:59.802141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:18:59.803580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.855808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d09c41d6ea14efaa98f4a964ccc928b', 'request_id': 'req-9c1fc018-9b33-4b52-aed2-51a58d793ca3'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.867785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d09c41d6ea14efaa98f4a964ccc928b', 'request_id': 'req-9c1fc018-9b33-4b52-aed2-51a58d793ca3'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.876305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d09c41d6ea14efaa98f4a964ccc928b', 'request_id': 'req-9c1fc018-9b33-4b52-aed2-51a58d793ca3'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.888991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] The token cache is being invalidate because project 2d09c41d6ea14efaa98f4a964ccc928b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:18:59.889288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.890509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9c1fc018-9b33-4b52-aed2-51a58d793ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:18:59.894436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1008/2009] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:18:59 2022] DELETE /identity/v3/projects/2d09c41d6ea14efaa98f4a964ccc928b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:18:59.899448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632776e-2f25-4f32-9b97-3f9317faed47 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:18:59.902907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132d50-d51f-4170-9701-0ae5a512e73c None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:18:59.902907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132d50-d51f-4170-9701-0ae5a512e73c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:18:59.902907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132d50-d51f-4170-9701-0ae5a512e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:18:59.923228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-07132d50-d51f-4170-9701-0ae5a512e73c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:18:59.924713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-07132d50-d51f-4170-9701-0ae5a512e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:18:59.969838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:00.014879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1009/2010] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:00.024212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.025198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110283ee-2431-4744-b66a-41f97fecceef None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.025491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110283ee-2431-4744-b66a-41f97fecceef None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.025765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110283ee-2431-4744-b66a-41f97fecceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.026331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.026615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.028200 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.033809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.060876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-110283ee-2431-4744-b66a-41f97fecceef None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:00.062094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-110283ee-2431-4744-b66a-41f97fecceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.063665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.065094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.072342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1010/2011] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:00.077747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110283ee-2431-4744-b66a-41f97fecceef None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.078792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1002/2012] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:00.090988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110283ee-2431-4744-b66a-41f97fecceef None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.098244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.099798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.100158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.100506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.101381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:00.102123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.104205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.124214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1011/2013] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:00.125548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.140944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5346686-2409-4e88-81c6-092d3ce283c7 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.147162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.147947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.148433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.148591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.149554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.151711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.184764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1003/2014] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:00 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:00.185872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.196289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec4520-4589-4235-8b6f-001ed7d5f441 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.291734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.292907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.293112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.293704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.294453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.296468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.323409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1012/2015] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:00 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:00.327548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.339763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043acee7-1cf9-44d8-b7f3-6238874476fc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.345218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.346139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.346883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.347627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.347943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.350165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.362196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1004/2016] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:00.367403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.393325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69897f82-6964-47f4-9a38-780878a491c6 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.399380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.400211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.400885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.401256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.402077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.403884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.419786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1013/2017] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:00.424192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.440207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78881d35-3f86-4bec-b1d3-77bd1f9c7dc7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.448922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792bd9f-ba06-4d7d-b8d4-1df8dcc5c990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.449948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792bd9f-ba06-4d7d-b8d4-1df8dcc5c990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.450307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792bd9f-ba06-4d7d-b8d4-1df8dcc5c990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.450637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792bd9f-ba06-4d7d-b8d4-1df8dcc5c990 admin admin] PATH_INFO: `/v3/projects/be0d2383b2c646ecb6bcfa1390a71f92/users/2f99cd2f163e4b58a0f4666039a07c62/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.470334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.472844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.488802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792bd9f-ba06-4d7d-b8d4-1df8dcc5c990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0d2383b2c646ecb6bcfa1390a71f92, user_id=2f99cd2f163e4b58a0f4666039a07c62, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.492141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792bd9f-ba06-4d7d-b8d4-1df8dcc5c990 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.496002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658c103-ee59-4dbc-bb6f-b88bfae453a2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.503551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.506289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.506582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.506900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] PATH_INFO: `/v3/projects/b79c7eab7edf4beda1efb340d2869a31` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.510404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79c7eab7edf4beda1efb340d2869a31)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.512884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.513736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1005/2018] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:00 2022] PUT /identity/v3/projects/be0d2383b2c646ecb6bcfa1390a71f92/users/2f99cd2f163e4b58a0f4666039a07c62/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:00.524805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.525310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.525600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.587092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] MFA Rules not processed for user `2f99cd2f163e4b58a0f4666039a07c62`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:00.589666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.590694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79c7eab7edf4beda1efb340d2869a31', 'request_id': 'req-f0d858dc-5852-491d-b05a-98ed5eae7b31'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.602565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79c7eab7edf4beda1efb340d2869a31', 'request_id': 'req-f0d858dc-5852-491d-b05a-98ed5eae7b31'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.619381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79c7eab7edf4beda1efb340d2869a31', 'request_id': 'req-f0d858dc-5852-491d-b05a-98ed5eae7b31'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.638702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:00.643233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] The token cache is being invalidate because project b79c7eab7edf4beda1efb340d2869a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:00.644048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.645130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.647579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1014/2019] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:00 2022] DELETE /identity/v3/projects/b79c7eab7edf4beda1efb340d2869a31 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:00.651225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.652943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.685809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1006/2020] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:00.690610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.692876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.698264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:00.747824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d858dc-5852-491d-b05a-98ed5eae7b31 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.752548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.753600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.753955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.754296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] PATH_INFO: `/v3/users/f1b697a54294423e9126e7c822c65f1b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.756284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b697a54294423e9126e7c822c65f1b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.758597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.762259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c215f4-e682-4b4b-9acc-e810aad1b092 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.767059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.767721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.768102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.768385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.769099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.771791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.800441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1007/2021] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:00 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:00.801477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.805544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b697a54294423e9126e7c822c65f1b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.806312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b697a54294423e9126e7c822c65f1b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.814451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b697a54294423e9126e7c822c65f1b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.815489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b697a54294423e9126e7c822c65f1b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.827807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b697a54294423e9126e7c822c65f1b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:00.840624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1015/2022] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:00 2022] DELETE /identity/v3/users/f1b697a54294423e9126e7c822c65f1b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:00.863698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825b205-614c-49c1-827b-a92e8c09f308 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.925075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.925828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.926213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.927228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.928174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.930177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.939350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:00.954518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1008/2023] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:00 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:00.955838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:00.957331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:00.971025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f0402-4447-4b4d-8c1b-d3532501ee26 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:00.976570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:00.977420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:00.977706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:00.977981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:00.978736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:00.980366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:00.991728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1016/2024] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:00.995833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.006345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed5f9d-070a-4f44-befa-a598e38ef295 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.011170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.012054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.012342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.012620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] PATH_INFO: `/v3/projects/4b6799ab2a314cc7b33193a8fcb5c2cc/users/8812211558b141d084b49a5ca4dbec4a/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.028735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6799ab2a314cc7b33193a8fcb5c2cc, user_id=8812211558b141d084b49a5ca4dbec4a, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.031435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.046375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1009/2025] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:00 2022] PUT /identity/v3/projects/4b6799ab2a314cc7b33193a8fcb5c2cc/users/8812211558b141d084b49a5ca4dbec4a/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:01.050365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.062005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977acd0-4d22-4959-b0fd-4743f097d38b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.126603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.128775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.163284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.164215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.164654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.165583 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.166343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.168149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.185936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1017/2026] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:01.190928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.203972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb14869-4610-44b0-bd84-2fc7aa60513c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.211970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207041c0-a49e-4eca-aea9-380aad281bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.213111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207041c0-a49e-4eca-aea9-380aad281bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.213580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207041c0-a49e-4eca-aea9-380aad281bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.214069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207041c0-a49e-4eca-aea9-380aad281bf3 admin admin] PATH_INFO: `/v3/projects/4b6799ab2a314cc7b33193a8fcb5c2cc/users/8812211558b141d084b49a5ca4dbec4a/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.216237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207041c0-a49e-4eca-aea9-380aad281bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6799ab2a314cc7b33193a8fcb5c2cc, user_id=8812211558b141d084b49a5ca4dbec4a, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.219168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207041c0-a49e-4eca-aea9-380aad281bf3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.225783 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bb7d5-7fbd-4043-908a-3c268d7ca41b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.230857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.231811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.232164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.232525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] PATH_INFO: `/v3/projects/0958d610c9d3405183b2d30452e43dba` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.234150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0958d610c9d3405183b2d30452e43dba)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.236696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.241760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1018/2027] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:01 2022] PUT /identity/v3/projects/4b6799ab2a314cc7b33193a8fcb5c2cc/users/8812211558b141d084b49a5ca4dbec4a/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:01.263049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.263049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.263049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.291482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:01.299185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] MFA Rules not processed for user `8812211558b141d084b49a5ca4dbec4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:01.301547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.324139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0958d610c9d3405183b2d30452e43dba', 'request_id': 'req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.335839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0958d610c9d3405183b2d30452e43dba', 'request_id': 'req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.344173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0958d610c9d3405183b2d30452e43dba', 'request_id': 'req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.358688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] The token cache is being invalidate because project 0958d610c9d3405183b2d30452e43dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:01.359266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.360397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e7ad6ef0-8d7c-4d95-8034-2c5d840c579c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.362768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1010/2028] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:01 2022] DELETE /identity/v3/projects/0958d610c9d3405183b2d30452e43dba => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:01.369258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12e0aa-e70c-4a8f-9a06-eb6a1499b062 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.369775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12e0aa-e70c-4a8f-9a06-eb6a1499b062 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.370419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12e0aa-e70c-4a8f-9a06-eb6a1499b062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.387802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1019/2029] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:01.392111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bf12e0aa-e70c-4a8f-9a06-eb6a1499b062 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:01.393316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bf12e0aa-e70c-4a8f-9a06-eb6a1499b062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.399312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.400848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.460048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:01.465007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d4e9f-5029-4730-9469-3f34c0aabe4f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.468870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.469611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.469959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.470293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.475079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1011/2030] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:01.481101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.482624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.484231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.484713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.485213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.493102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1020/2031] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:01.503300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:01.504496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.513130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1012/2032] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:01.521966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.541587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab38f5c-5ad6-47d1-a9a0-297ffc0fe6b1 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.546459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.547218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.547713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.547997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.548933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:01.549278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.551059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.567952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1021/2033] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:19:01.572352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.584492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a9bcc-82db-4c0d-856d-f9f36b4b1652 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.591112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.592884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.593300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.593589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.594310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.596526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.635928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1013/2034] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:01 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:01.643612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.644163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.647976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.657714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca33f8f-46b0-492e-b013-8aba5cbc470c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.730522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256ae6-fe7e-43fe-8be8-9f4ceae93cca admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.735807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.737437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.737848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.738202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] PATH_INFO: `/v3/users/2f99cd2f163e4b58a0f4666039a07c62` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.739544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f99cd2f163e4b58a0f4666039a07c62)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.740201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.741734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.742408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.742700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.743017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.743931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.746259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.776709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1022/2035] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:01 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:01.781166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.796715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117164c-ad2a-4696-bd33-8c243a4aa188 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.800416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f99cd2f163e4b58a0f4666039a07c62'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.801168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f99cd2f163e4b58a0f4666039a07c62'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.802249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.803090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.803377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.803982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.805127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.807212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.809291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f99cd2f163e4b58a0f4666039a07c62'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.810185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f99cd2f163e4b58a0f4666039a07c62'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.819954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fb31cf61-7413-44f1-8883-2511ac549b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f99cd2f163e4b58a0f4666039a07c62'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:01.823770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1023/2036] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:01.824741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.826673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.840691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2977686-044c-4d6c-a362-037244ba381c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.846795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1014/2037] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:01 2022] DELETE /identity/v3/users/2f99cd2f163e4b58a0f4666039a07c62 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:01.849726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.849726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.849726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.849726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.850983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.851829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.852207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.852576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.852720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:01.854274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:01.871833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1024/2038] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:01.875842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:01.877324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.879144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.890058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edcc33-ed99-44ad-9254-5f04ac341a49 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:01.944854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:01.966455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1015/2039] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:01.966697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda661ca-2cdd-4ba9-b4a4-c8d68cb3bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:01.967681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda661ca-2cdd-4ba9-b4a4-c8d68cb3bfae admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:01.967963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda661ca-2cdd-4ba9-b4a4-c8d68cb3bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:01.968239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda661ca-2cdd-4ba9-b4a4-c8d68cb3bfae admin admin] PATH_INFO: `/v3/projects/eba299ed86a642a2bc7a3b78c5d1bd4c/users/cc417000fcd244acac24aa894eb50d55/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:01.976809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:01.978166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:01.983386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda661ca-2cdd-4ba9-b4a4-c8d68cb3bfae admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba299ed86a642a2bc7a3b78c5d1bd4c, user_id=cc417000fcd244acac24aa894eb50d55, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:01.986024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda661ca-2cdd-4ba9-b4a4-c8d68cb3bfae admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.000874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1025/2040] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:01 2022] PUT /identity/v3/projects/eba299ed86a642a2bc7a3b78c5d1bd4c/users/cc417000fcd244acac24aa894eb50d55/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:02.012050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.012345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.012754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.040414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] MFA Rules not processed for user `cc417000fcd244acac24aa894eb50d55`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:02.041681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:02.044352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005591b2-b619-4cc8-9c5c-9893c0da162c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.095068 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.095721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.096070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.097535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.106479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1026/2041] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:02.110901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.158184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.159667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.170668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1016/2042] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:02.183567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee67cb-c3f7-4b09-b2cc-32a804ee1d78 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.189150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.189838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.190109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.190385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.191157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.193895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.207436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:02.222965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1027/2043] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:02 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:02.227374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.237452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3e3aa-f620-49c6-beb9-423288540e88 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.305079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:02.316691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.318055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.318351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.318645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.319485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.321728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.349512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1017/2044] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:02 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:02.353436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.364910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b189aac-49cc-4b39-b184-061b2ae120a9 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.369919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.370585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.370924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.371180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.371922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.373594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.385328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1028/2045] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:02.389615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.399753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1603-01b1-4b4a-a0a2-69be03f78ed2 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.404643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.405536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.405819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.406095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] PATH_INFO: `/v3/projects/aeb1ad6691fc4d68bde7256b2806ae72/users/a49fcaa8aa4f4aa9b130087e3b797d38/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.423189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1ad6691fc4d68bde7256b2806ae72, user_id=a49fcaa8aa4f4aa9b130087e3b797d38, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.425882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.449726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1018/2046] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:02 2022] PUT /identity/v3/projects/aeb1ad6691fc4d68bde7256b2806ae72/users/a49fcaa8aa4f4aa9b130087e3b797d38/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:02.451027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.462183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48b80e-2080-4a97-b23f-2288f9393ada admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.493608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.496753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:02.526932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.527755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.528159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.528551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.529525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.531274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.547430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1029/2047] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:02.548888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.560807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28cce3-aa5b-4ce7-b0db-bd48a23ee5d4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.566358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3823cdd-cea3-4585-afef-1eb8e49cc142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.567324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3823cdd-cea3-4585-afef-1eb8e49cc142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.567610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3823cdd-cea3-4585-afef-1eb8e49cc142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.567891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3823cdd-cea3-4585-afef-1eb8e49cc142 admin admin] PATH_INFO: `/v3/projects/aeb1ad6691fc4d68bde7256b2806ae72/users/a49fcaa8aa4f4aa9b130087e3b797d38/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.572783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3823cdd-cea3-4585-afef-1eb8e49cc142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1ad6691fc4d68bde7256b2806ae72, user_id=a49fcaa8aa4f4aa9b130087e3b797d38, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.576007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3823cdd-cea3-4585-afef-1eb8e49cc142 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.587076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1481f5-8f79-4512-9b2b-56bd27d2c8b6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.588565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1030/2048] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:02 2022] PUT /identity/v3/projects/aeb1ad6691fc4d68bde7256b2806ae72/users/a49fcaa8aa4f4aa9b130087e3b797d38/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:02.599218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.599511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.599791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.634380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] MFA Rules not processed for user `a49fcaa8aa4f4aa9b130087e3b797d38`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:02.635630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:02.650756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.651658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.651972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.652254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] PATH_INFO: `/v3/projects/be0d2383b2c646ecb6bcfa1390a71f92` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.653445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0d2383b2c646ecb6bcfa1390a71f92)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.655419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.706381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0d2383b2c646ecb6bcfa1390a71f92', 'request_id': 'req-872cf30f-615b-4c7a-ba30-25dc17d1003e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.714250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1031/2049] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:02.714407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0d2383b2c646ecb6bcfa1390a71f92', 'request_id': 'req-872cf30f-615b-4c7a-ba30-25dc17d1003e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.722343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0d2383b2c646ecb6bcfa1390a71f92', 'request_id': 'req-872cf30f-615b-4c7a-ba30-25dc17d1003e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.725355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.727009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:02.731189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:02.735429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] The token cache is being invalidate because project be0d2383b2c646ecb6bcfa1390a71f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:02.736164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.737649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.739999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1019/2050] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:02 2022] DELETE /identity/v3/projects/be0d2383b2c646ecb6bcfa1390a71f92 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:02.743878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:02.746180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:02.824760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cf30f-615b-4c7a-ba30-25dc17d1003e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.826764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265cb4b-5d3f-4691-b08f-0b1308de452b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:02.830026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.830777 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:02.831171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.831456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.832326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] PATH_INFO: `/v3/users/8812211558b141d084b49a5ca4dbec4a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.835323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8812211558b141d084b49a5ca4dbec4a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.837582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:02.897350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:02.898170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:02.898508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:02.899034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:02.900979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:02.907687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8812211558b141d084b49a5ca4dbec4a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.909029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8812211558b141d084b49a5ca4dbec4a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.919493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8812211558b141d084b49a5ca4dbec4a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.920191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8812211558b141d084b49a5ca4dbec4a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.928386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8812211558b141d084b49a5ca4dbec4a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:02.948554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1020/2051] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:02 2022] DELETE /identity/v3/users/8812211558b141d084b49a5ca4dbec4a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:02.987983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:02.990281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:03.004319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1032/2052] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:03.103193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:03.105154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:03.186326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d899-8c04-4dbd-aeb2-8748114ed989 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:03.190754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:03.191456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.191750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.192231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.194117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:03.235341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:03.269567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:03.271345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b838c-38af-4ba8-8286-7e0a9c3e3402 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:03.282506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1021/2053] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:03.414626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:03.429437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3f3cc-4340-4267-bc50-54530ada5ed8 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:03.530910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:03.531888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.532275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.532663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] PATH_INFO: `/v3/projects/4b6799ab2a314cc7b33193a8fcb5c2cc` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.534334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6799ab2a314cc7b33193a8fcb5c2cc)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:03.537685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:03.603701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6799ab2a314cc7b33193a8fcb5c2cc', 'request_id': 'req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:03.613394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6799ab2a314cc7b33193a8fcb5c2cc', 'request_id': 'req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:03.617908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:03.627338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6799ab2a314cc7b33193a8fcb5c2cc', 'request_id': 'req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:03.644220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] The token cache is being invalidate because project 4b6799ab2a314cc7b33193a8fcb5c2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:03.644668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:03.645874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3be86aa2-6da9-4134-ae0b-12fbcf6a7279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:03.648172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1033/2054] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:03 2022] DELETE /identity/v3/projects/4b6799ab2a314cc7b33193a8fcb5c2cc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:03.652793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.653207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.653345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.674599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:03.675945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:03.745757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1022/2055] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:03.752842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.753208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.753557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.776162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:03.777395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:03.785560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1034/2056] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:03.790247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:03.800685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6ac81-86f0-46a5-b6c6-ca2641292c8c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:03.805255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:03.805917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.806202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.806478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.807246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:03.807608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:03.809375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:03.822600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1023/2057] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:03.827095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:03.837085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd88fe-9d58-47df-befb-f7faa4435adc None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:03.841986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:03.842631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.842938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.843213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.843924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:03.845815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:03.870233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1035/2058] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:03 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:03.874284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:03.884562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb83b2-a717-4ea2-994b-a3cb8235703e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:03.946661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:03.947417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.947700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.948319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.949055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:03.951333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:03.974628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1024/2059] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:03 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:03.978957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:03.989643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e75118-6d68-4d5c-a50c-d28f1c9b1239 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:03.995919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:03.996591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:03.996874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:03.997303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:03.998060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.000138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.022933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1036/2060] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:04.027158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.039595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a96b-d005-45ed-bba2-ca8bc320ce6d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.044418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008478ec-56d1-4c51-84fb-845b834cea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.045081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008478ec-56d1-4c51-84fb-845b834cea26 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.045411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008478ec-56d1-4c51-84fb-845b834cea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.045692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008478ec-56d1-4c51-84fb-845b834cea26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.046421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008478ec-56d1-4c51-84fb-845b834cea26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.048114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008478ec-56d1-4c51-84fb-845b834cea26 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.060999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1025/2061] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:04.066570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.083518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778b24e-8a10-4ee4-aeae-d9f8b13ab8aa admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.089164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.090215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.090667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.090979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] PATH_INFO: `/v3/projects/2b990df3465f420da8f806da92600d82/users/87fd9305e1194ef19708230f3ad107f9/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.107127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b990df3465f420da8f806da92600d82, user_id=87fd9305e1194ef19708230f3ad107f9, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.109869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.127444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1037/2062] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:04 2022] PUT /identity/v3/projects/2b990df3465f420da8f806da92600d82/users/87fd9305e1194ef19708230f3ad107f9/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:04.139646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.140198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.140545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.168757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] MFA Rules not processed for user `87fd9305e1194ef19708230f3ad107f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:04.169984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:04.199142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:04.234636 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1026/2063] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:04.239723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.300185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695ebfc-16ca-4449-aa54-fa6f5c2eee14 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.305342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.305993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.306276 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.306558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.307278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.309170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.333706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1038/2064] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:04 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:04.337704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.351354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239a8b9-7b2b-4590-be78-b44a68435974 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.412429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.413000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.413287 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.413566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.414269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.416228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.446784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1027/2065] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:04 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:04.451014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.463422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acf1c4-9e47-4b93-8e9a-faef5c65508d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.472655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.473852 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.474811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.475200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.476927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.479313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.507863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.508089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1039/2066] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:04.521958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2aea-b4ab-4d44-a90c-892843ced2b5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.529294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.530427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.534440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.534440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] PATH_INFO: `/v3/projects/bba8fdd2698f418b8c3c2640ed2f07e3/users/56f904b584cf470a8560289cbaa75096/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.551255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba8fdd2698f418b8c3c2640ed2f07e3, user_id=56f904b584cf470a8560289cbaa75096, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.554907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.572153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1028/2067] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:04 2022] PUT /identity/v3/projects/bba8fdd2698f418b8c3c2640ed2f07e3/users/56f904b584cf470a8560289cbaa75096/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:04.578854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.590494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fb572-5545-4d3a-8a38-bd5c4f431121 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.659975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:04.668206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031dd8ad-575f-42d5-a0f8-991cf74b3a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.668969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dd8ad-575f-42d5-a0f8-991cf74b3a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.669364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dd8ad-575f-42d5-a0f8-991cf74b3a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.669753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031dd8ad-575f-42d5-a0f8-991cf74b3a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.670513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dd8ad-575f-42d5-a0f8-991cf74b3a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.672789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031dd8ad-575f-42d5-a0f8-991cf74b3a89 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.701537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1040/2068] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:04.702585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.715274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc7679-eed9-42bb-9499-ee2b53391367 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.720914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.722103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.722543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.722857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] PATH_INFO: `/v3/projects/bba8fdd2698f418b8c3c2640ed2f07e3/users/56f904b584cf470a8560289cbaa75096/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.727676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba8fdd2698f418b8c3c2640ed2f07e3, user_id=56f904b584cf470a8560289cbaa75096, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.733582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:04.748381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1029/2069] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:04 2022] PUT /identity/v3/projects/bba8fdd2698f418b8c3c2640ed2f07e3/users/56f904b584cf470a8560289cbaa75096/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:04.761831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.762169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.762470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.797850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] MFA Rules not processed for user `56f904b584cf470a8560289cbaa75096`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:04.799422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:04.827615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.829397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:04.921062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1041/2070] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:04.937012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8a47-d749-4b54-b2f4-1ff08d76a915 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:04.943168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:04.944397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:04.944696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:04.945041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] PATH_INFO: `/v3/users/5cc7a6d51eab4706827d125415c63a6b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:04.946439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc7a6d51eab4706827d125415c63a6b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:04.948287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:04.949742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:04.957211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:05.004115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:05.047073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68346fc-62e8-4166-8fb0-daf268e2d930 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:05.050176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc7a6d51eab4706827d125415c63a6b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.052263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc7a6d51eab4706827d125415c63a6b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.056074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ef345-f3de-4e43-831b-bc09ac954d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:05.056740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ef345-f3de-4e43-831b-bc09ac954d9e service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:05.057024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ef345-f3de-4e43-831b-bc09ac954d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:05.057298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ef345-f3de-4e43-831b-bc09ac954d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:05.065549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc7a6d51eab4706827d125415c63a6b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.066412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc7a6d51eab4706827d125415c63a6b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.070604 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ef345-f3de-4e43-831b-bc09ac954d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:05.072720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ef345-f3de-4e43-831b-bc09ac954d9e service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:05.078070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc7a6d51eab4706827d125415c63a6b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.085170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1042/2071] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:05.137886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1030/2072] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:04 2022] DELETE /identity/v3/users/5cc7a6d51eab4706827d125415c63a6b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:05.147829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4773-e003-4386-a33f-ce6197581477 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:05.148209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4773-e003-4386-a33f-ce6197581477 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:05.148625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a4773-e003-4386-a33f-ce6197581477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:05.177188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-212a4773-e003-4386-a33f-ce6197581477 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:05.178845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-212a4773-e003-4386-a33f-ce6197581477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:05.272472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1043/2073] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:05.280037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:05.281818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:05.331575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4773-e003-4386-a33f-ce6197581477 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:05.333374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-212a4773-e003-4386-a33f-ce6197581477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:05.367091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:05.367266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae2709-4e10-48ea-a1b0-95e5674c4c1a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:05.371587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:05.372311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:05.372666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:05.373006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:05.387113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:05.388597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:05.400163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1031/2074] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:05.410147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a4773-e003-4386-a33f-ce6197581477 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:05.414672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:05.415428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:05.415712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:05.415979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:05.423917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:05.510275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:05.512000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:05.523988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1044/2075] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:05.729147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:05.730793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:05.745944 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21c414-5296-469e-8736-82f9c8763da5 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:05.751691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:05.752936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:05.752936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:05.753145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] PATH_INFO: `/v3/projects/501b8e05b7a942df9c7a748d8be3c965` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:05.754264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=501b8e05b7a942df9c7a748d8be3c965)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:05.756910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:05.811260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501b8e05b7a942df9c7a748d8be3c965', 'request_id': 'req-8f69b1b2-c469-4d54-821e-fed0a1bf4940'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.818653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501b8e05b7a942df9c7a748d8be3c965', 'request_id': 'req-8f69b1b2-c469-4d54-821e-fed0a1bf4940'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.828108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501b8e05b7a942df9c7a748d8be3c965', 'request_id': 'req-8f69b1b2-c469-4d54-821e-fed0a1bf4940'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.840926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] The token cache is being invalidate because project 501b8e05b7a942df9c7a748d8be3c965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:05.841216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.841870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.843668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1032/2076] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:05 2022] DELETE /identity/v3/projects/501b8e05b7a942df9c7a748d8be3c965 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:05.847774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:05.859811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cdad9-cc3a-4aca-8525-11d64715c09b service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:05.922842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:05.923825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:05.924104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:05.924376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] PATH_INFO: `/v3/users/a7fe1c957bde4ec6b3360d91a98910b2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:05.925553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fe1c957bde4ec6b3360d91a98910b2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:05.928022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:05.974183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fe1c957bde4ec6b3360d91a98910b2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.974839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fe1c957bde4ec6b3360d91a98910b2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.981808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fe1c957bde4ec6b3360d91a98910b2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.982472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fe1c957bde4ec6b3360d91a98910b2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.994676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-841f51ab-ed60-4df0-84ca-ef93383108da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fe1c957bde4ec6b3360d91a98910b2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:05.998259 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:06.008938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1045/2077] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:05 2022] DELETE /identity/v3/users/a7fe1c957bde4ec6b3360d91a98910b2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:06.275439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.276905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:06.292042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69b1b2-c469-4d54-821e-fed0a1bf4940 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.317566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:06.353745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:06.354611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.354926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.355207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] PATH_INFO: `/v3/projects/ec8bf6f1c8de4be391d75d4a8d00c6bd` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.356371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8bf6f1c8de4be391d75d4a8d00c6bd)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:06.358271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:06.405415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8bf6f1c8de4be391d75d4a8d00c6bd', 'request_id': 'req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.412660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8bf6f1c8de4be391d75d4a8d00c6bd', 'request_id': 'req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.420262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8bf6f1c8de4be391d75d4a8d00c6bd', 'request_id': 'req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.434679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] The token cache is being invalidate because project ec8bf6f1c8de4be391d75d4a8d00c6bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:06.435041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.435720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dcea18d0-5eef-4a31-ad18-4dcf56a0c8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.437538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1033/2078] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:06 2022] DELETE /identity/v3/projects/ec8bf6f1c8de4be391d75d4a8d00c6bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:06.443075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.443412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.443738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.462736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:06.463977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:06.529409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1046/2079] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:06.539666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.539942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.540227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.583920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:06.584845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:06.607351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1034/2080] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:06.609349 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.623747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22a48d-142d-4148-9b68-9b4b4cc8de1a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.628535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:06.629197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.629480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.629800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.630537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:06.630855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:06.632475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:06.645195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1047/2081] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:06.650202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.664208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2809d7-2e52-45ec-8362-df78dc2986c8 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.675861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:06.679325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.679682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.679937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.680816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:06.683589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:06.707130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.710279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:06.727726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1035/2082] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:06 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:06.728028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6d589-b7dc-4e7e-ade0-ba82800fcf4e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.731311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.806358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:06.807394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.807687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.807967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] PATH_INFO: `/v3/users/cc417000fcd244acac24aa894eb50d55` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.809898 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc417000fcd244acac24aa894eb50d55)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:06.813611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:06.828580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22736eab-7b72-4014-af8d-e3b99146b69b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.835942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:06.837772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.838264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.838743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.840580 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:06.842727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:06.877255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc417000fcd244acac24aa894eb50d55'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.878240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1036/2083] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:06 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:06.878765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc417000fcd244acac24aa894eb50d55'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.882933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.887561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc417000fcd244acac24aa894eb50d55'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.888446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc417000fcd244acac24aa894eb50d55'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.900507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b27277-6a49-4b46-9d0b-110023c708af admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.907498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:06.908540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.909039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.909773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.910742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:06.911669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-125fada1-462c-4aec-965e-817b7749d325 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc417000fcd244acac24aa894eb50d55'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:06.912685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:06.925939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1037/2084] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:06.929354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:06.931491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:06.935477 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1048/2085] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:06 2022] DELETE /identity/v3/users/cc417000fcd244acac24aa894eb50d55 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:06.937132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:06.937475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:06.938572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:06.949025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:06.961646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d920a-22ef-40de-82dd-f4893a9511b3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:06.972647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:06.972647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:06.976632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:07.046298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1049/2086] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:07.055752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:07.057215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.057438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.057714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.058451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:07.060402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:07.073419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1038/2087] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:07.077306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:07.087831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc79a1-dc2c-4de6-bb20-dbbb8cbddd3b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:07.093116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270111e9-0910-4e71-8fdf-8fd2640bad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:07.094133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270111e9-0910-4e71-8fdf-8fd2640bad74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.094288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270111e9-0910-4e71-8fdf-8fd2640bad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.094552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270111e9-0910-4e71-8fdf-8fd2640bad74 admin admin] PATH_INFO: `/v3/projects/7b326a70421a4b33b8cd5dfda7bcad69/users/a3dd7d98f171470fb6e90e5ed199bdda/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.101278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:07.103151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.131133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270111e9-0910-4e71-8fdf-8fd2640bad74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b326a70421a4b33b8cd5dfda7bcad69, user_id=a3dd7d98f171470fb6e90e5ed199bdda, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:07.135541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270111e9-0910-4e71-8fdf-8fd2640bad74 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:07.187908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1050/2088] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:07 2022] PUT /identity/v3/projects/7b326a70421a4b33b8cd5dfda7bcad69/users/a3dd7d98f171470fb6e90e5ed199bdda/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:07.190100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c159c-9bb5-4855-8cdb-3d243be1aeeb None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.190404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c159c-9bb5-4855-8cdb-3d243be1aeeb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.190687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c159c-9bb5-4855-8cdb-3d243be1aeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.192956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dba39-d52a-4605-88c2-02ff23c37607 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:07.234686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4c0c159c-9bb5-4855-8cdb-3d243be1aeeb None None] MFA Rules not processed for user `a3dd7d98f171470fb6e90e5ed199bdda`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:07.237098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4c0c159c-9bb5-4855-8cdb-3d243be1aeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.270127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:07.271901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.272576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.273034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.359808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:07.361672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:07.376789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1039/2089] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:07.404843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1051/2090] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:07.418981 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:07.421188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.436359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:07.498192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad800e-d15d-4343-a40f-fbe92e2c9d2c service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:07.503149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fbef6-ba7e-4eec-acb2-ed32aece3948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:07.503149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fbef6-ba7e-4eec-acb2-ed32aece3948 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.503365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fbef6-ba7e-4eec-acb2-ed32aece3948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.503985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fbef6-ba7e-4eec-acb2-ed32aece3948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.514482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fbef6-ba7e-4eec-acb2-ed32aece3948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:07.515938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fbef6-ba7e-4eec-acb2-ed32aece3948 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:07.526998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1040/2091] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:07.565095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97344a-c6b2-4945-8334-ca0369c7b680 None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.565618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97344a-c6b2-4945-8334-ca0369c7b680 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.565618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97344a-c6b2-4945-8334-ca0369c7b680 None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.569351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1052/2092] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:19:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:19:07.578088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.578385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.578654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.610321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] MFA Rules not processed for user `702fa2f56dab462ea0c644ddb516ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:07.613302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.642296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97344a-c6b2-4945-8334-ca0369c7b680 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:07.644165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2c97344a-c6b2-4945-8334-ca0369c7b680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.679432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97344a-c6b2-4945-8334-ca0369c7b680 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:07.695125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:07.697355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.697656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.697925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] PATH_INFO: `/v3/users/106d1325381147b1ba157f2d54d0388c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.701246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=106d1325381147b1ba157f2d54d0388c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:07.705860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:07.785480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1041/2093] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:07.794342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:07.797005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.801391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '106d1325381147b1ba157f2d54d0388c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:07.802482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '106d1325381147b1ba157f2d54d0388c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:07.816687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '106d1325381147b1ba157f2d54d0388c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:07.818158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '106d1325381147b1ba157f2d54d0388c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:07.825923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:07.836568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6db484-74ea-4ea3-80a6-5905e2b668f1 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:07.849552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '106d1325381147b1ba157f2d54d0388c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:07.853546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:07.854231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:07.854524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:07.856271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:07.859473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:07.869317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1053/2094] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:07 2022] DELETE /identity/v3/users/106d1325381147b1ba157f2d54d0388c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:07.882543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:07.884775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:07.898101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1042/2095] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:08.247832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:08.249362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.266038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:08.268804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.294808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca76c3-2d4b-41f3-b26f-3b85befe6e0b service placement] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:08.373493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab70f00-053d-41f5-aeb3-1253a7dd3f9a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:08.378857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:08.380584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.380874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.381160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] PATH_INFO: `/v3/projects/eba299ed86a642a2bc7a3b78c5d1bd4c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.382395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:08.382534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba299ed86a642a2bc7a3b78c5d1bd4c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:08.384624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:08.391390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.391777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.392208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] PATH_INFO: `/v3/projects/7b95f79de5724987ab6c9fddb86a84f0` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.394470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b95f79de5724987ab6c9fddb86a84f0)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:08.397148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:08.472901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:08.476380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba299ed86a642a2bc7a3b78c5d1bd4c', 'request_id': 'req-8419463e-e8dd-4bc1-9956-a0891bb8874f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.495381 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b95f79de5724987ab6c9fddb86a84f0', 'request_id': 'req-244c0795-21a6-46be-be0c-a59c60d77416'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.504296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba299ed86a642a2bc7a3b78c5d1bd4c', 'request_id': 'req-8419463e-e8dd-4bc1-9956-a0891bb8874f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.510117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b95f79de5724987ab6c9fddb86a84f0', 'request_id': 'req-244c0795-21a6-46be-be0c-a59c60d77416'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.515074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba299ed86a642a2bc7a3b78c5d1bd4c', 'request_id': 'req-8419463e-e8dd-4bc1-9956-a0891bb8874f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.523284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b95f79de5724987ab6c9fddb86a84f0', 'request_id': 'req-244c0795-21a6-46be-be0c-a59c60d77416'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.544578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] The token cache is being invalidate because project eba299ed86a642a2bc7a3b78c5d1bd4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:08.545013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.545744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.551741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1054/2096] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:08 2022] DELETE /identity/v3/projects/eba299ed86a642a2bc7a3b78c5d1bd4c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:08.552787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:08.557501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.557653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] The token cache is being invalidate because project 7b95f79de5724987ab6c9fddb86a84f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:08.557653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.557653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-244c0795-21a6-46be-be0c-a59c60d77416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.559356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1043/2097] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:08 2022] DELETE /identity/v3/projects/7b95f79de5724987ab6c9fddb86a84f0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:08.574076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f439993-32d9-48b3-aaff-4b16a8b5dfb1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.574370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f439993-32d9-48b3-aaff-4b16a8b5dfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.574647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f439993-32d9-48b3-aaff-4b16a8b5dfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.594310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0f439993-32d9-48b3-aaff-4b16a8b5dfb1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:08.595456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0f439993-32d9-48b3-aaff-4b16a8b5dfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.665474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419463e-e8dd-4bc1-9956-a0891bb8874f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:08.686957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:08.694960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1044/2098] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:08.708302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.708659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.709055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.729496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:08.730951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.740217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1045/2099] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:08.744332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:08.752805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:08.758594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421a20d-1223-4b33-a6d0-e2faaea4214e None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:08.767464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.767464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.767464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] PATH_INFO: `/v3/users/a49fcaa8aa4f4aa9b130087e3b797d38` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.767464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49fcaa8aa4f4aa9b130087e3b797d38)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:08.767877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:08.768560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.769022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.769365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.770098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:08.770428 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:08.772073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:08.774951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:08.792431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1046/2100] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:08.794588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:08.797334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.836048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cfac-1725-4b2a-a42a-65cb6e600567 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:08.844172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:08.845852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:08.845852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:08.845852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] PATH_INFO: `/v3/users/5c76de7a9cd541eba48819222b5cfc07` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:08.847643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c76de7a9cd541eba48819222b5cfc07)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:08.849798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:08.867269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49fcaa8aa4f4aa9b130087e3b797d38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.868508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49fcaa8aa4f4aa9b130087e3b797d38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.896831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49fcaa8aa4f4aa9b130087e3b797d38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.898851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49fcaa8aa4f4aa9b130087e3b797d38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.922656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c76de7a9cd541eba48819222b5cfc07'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.923692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c76de7a9cd541eba48819222b5cfc07'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.928384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49fcaa8aa4f4aa9b130087e3b797d38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.932522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c76de7a9cd541eba48819222b5cfc07'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.933624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c76de7a9cd541eba48819222b5cfc07'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.947517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1055/2101] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:08 2022] DELETE /identity/v3/users/a49fcaa8aa4f4aa9b130087e3b797d38 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:08.948558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:08.949882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:08.955052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c76de7a9cd541eba48819222b5cfc07'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:08.988398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1047/2102] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:08 2022] DELETE /identity/v3/users/5c76de7a9cd541eba48819222b5cfc07 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:09.058011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e971703-ba47-45c9-ad9a-9a8c54be8c4b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.133443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:09.160922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.162858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.163333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.163850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.164783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.168154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.215349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.216199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1056/2103] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:08 2022] POST /identity/v3/projects => generated 389 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:09.217147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.237899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eec20c-6d32-4130-8dd9-3e3bbe7cf473 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.257516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:09.293114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.295550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.383971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d4dbf-5582-4fab-9411-b592256bf452 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.394015 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.394706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.394879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.395080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.396207 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.396564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.396912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.397352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.397733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] PATH_INFO: `/v3/projects/d7825374f809425688a5767b8c79e828` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.398879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.399311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7825374f809425688a5767b8c79e828)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.401401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.451662 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1048/2104] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:09 2022] POST /identity/v3/users => generated 295 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:09.452478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.454228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.481331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2fce7-4c4b-4339-b85f-c6057634e537 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.493758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7825374f809425688a5767b8c79e828', 'request_id': 'req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.502706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7825374f809425688a5767b8c79e828', 'request_id': 'req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.511225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7825374f809425688a5767b8c79e828', 'request_id': 'req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.524874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] The token cache is being invalidate because project d7825374f809425688a5767b8c79e828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:09.525241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.526585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.528263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1057/2105] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:09 2022] DELETE /identity/v3/projects/d7825374f809425688a5767b8c79e828 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:09.529027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.537531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.578523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.580555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.580840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.581220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] PATH_INFO: `/v3/projects/aeb1ad6691fc4d68bde7256b2806ae72` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.582559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb1ad6691fc4d68bde7256b2806ae72)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.586056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.626862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdeb0f-3afd-442a-9efc-2d5495c2d813 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.633000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.633685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.633961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.634250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.635034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.636641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.638771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:09.653815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1058/2106] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:09.655085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.656393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb1ad6691fc4d68bde7256b2806ae72', 'request_id': 'req-65970b5a-9c09-4280-a104-49869a6ec7b1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.658819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.665371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb1ad6691fc4d68bde7256b2806ae72', 'request_id': 'req-65970b5a-9c09-4280-a104-49869a6ec7b1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.675570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb1ad6691fc4d68bde7256b2806ae72', 'request_id': 'req-65970b5a-9c09-4280-a104-49869a6ec7b1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.681136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea63d4-b216-47a9-978c-3fd623f195b0 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.698656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] The token cache is being invalidate because project aeb1ad6691fc4d68bde7256b2806ae72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:09.699218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.700812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.703054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1049/2107] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:09 2022] DELETE /identity/v3/projects/aeb1ad6691fc4d68bde7256b2806ae72 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:09.704335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.705973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.754396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.755558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.755903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.756745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] PATH_INFO: `/v3/users/562ec6e2a9a34276ab4ade021eb0d33f` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.758517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562ec6e2a9a34276ab4ade021eb0d33f)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.761015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.772086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:09.812722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65970b5a-9c09-4280-a104-49869a6ec7b1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.818749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8532db-9f2c-4afd-a4fe-beec80f21d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:09.819462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8532db-9f2c-4afd-a4fe-beec80f21d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.819745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8532db-9f2c-4afd-a4fe-beec80f21d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.820023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8532db-9f2c-4afd-a4fe-beec80f21d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.820770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8532db-9f2c-4afd-a4fe-beec80f21d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:09.822382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8532db-9f2c-4afd-a4fe-beec80f21d87 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:09.832601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562ec6e2a9a34276ab4ade021eb0d33f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.834563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562ec6e2a9a34276ab4ade021eb0d33f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.838597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1050/2108] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:09.840063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.840486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.840771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.844467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562ec6e2a9a34276ab4ade021eb0d33f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.845380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562ec6e2a9a34276ab4ade021eb0d33f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.860496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ecb98768-502f-4cfd-8881-2b1eecc7db77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562ec6e2a9a34276ab4ade021eb0d33f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:09.874070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:09.875323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.886727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1051/2109] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:09.887512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:09.888780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:09.896639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1059/2110] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:09 2022] DELETE /identity/v3/users/562ec6e2a9a34276ab4ade021eb0d33f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:09.897772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:09.898077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:09.898337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:09.908186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fac05-f36b-4a6f-bffb-9878055215dd None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:09.918321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:09.919820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.012310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3232271-fb6f-4368-8276-60ba25ce262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.013485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232271-fb6f-4368-8276-60ba25ce262d admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.013928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232271-fb6f-4368-8276-60ba25ce262d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.014356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3232271-fb6f-4368-8276-60ba25ce262d admin admin] PATH_INFO: `/v3/projects/346889f9c6334aa9a22bd277aa002a0f/users/326e0d70244f46059664a135aefab0f0/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.045510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3232271-fb6f-4368-8276-60ba25ce262d admin admin] RBAC: Authorizing `identity:create_grant(project_id=346889f9c6334aa9a22bd277aa002a0f, user_id=326e0d70244f46059664a135aefab0f0, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.048635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3232271-fb6f-4368-8276-60ba25ce262d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.050029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1060/2111] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:10.054366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.068743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74841a0-c494-44e1-a6fd-a53f375eafb4 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.071916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1052/2112] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:09 2022] PUT /identity/v3/projects/346889f9c6334aa9a22bd277aa002a0f/users/326e0d70244f46059664a135aefab0f0/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:10.077604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.078020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.078412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.085193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:10.097598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:10.099250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.167203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.168299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.168422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.168769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.169538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:10.170600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.172541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.184810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1053/2113] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:10.187741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.189642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.190632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1061/2114] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:10.195865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.208600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f4909-d223-4bc8-8b36-3ee9485f18df admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.213671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51835825-dfc4-4c37-8e3c-9105aef57c4d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.213812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.215277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.215567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.216088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.216738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.218955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.226204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.227166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.227493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.227781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] PATH_INFO: `/v3/projects/831f4787578246b2a26996696ad59617` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.229189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=831f4787578246b2a26996696ad59617)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.231269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.253636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1062/2115] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:10.255008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.256696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.281637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:10.288323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831f4787578246b2a26996696ad59617', 'request_id': 'req-fc8006c7-f42b-484f-af63-8d5f5a091ecd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:10.297514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831f4787578246b2a26996696ad59617', 'request_id': 'req-fc8006c7-f42b-484f-af63-8d5f5a091ecd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:10.307338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831f4787578246b2a26996696ad59617', 'request_id': 'req-fc8006c7-f42b-484f-af63-8d5f5a091ecd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:10.318253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] The token cache is being invalidate because project 831f4787578246b2a26996696ad59617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:10.318624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:10.319353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:10.319907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ba821-87b8-4d20-a9c5-fb55302f2c7a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.321026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1054/2116] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:10 2022] DELETE /identity/v3/projects/831f4787578246b2a26996696ad59617 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:10.321960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.323382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.372177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.372830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.373115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.373393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.375347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.389505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8006c7-f42b-484f-af63-8d5f5a091ecd admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.394596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ac1db-2098-4537-8db1-3ed5e384a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.395859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ac1db-2098-4537-8db1-3ed5e384a03c admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.396151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ac1db-2098-4537-8db1-3ed5e384a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.396429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ac1db-2098-4537-8db1-3ed5e384a03c admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.397112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ac1db-2098-4537-8db1-3ed5e384a03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.399055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ac1db-2098-4537-8db1-3ed5e384a03c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.422010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1055/2117] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:10.426341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.426684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.426995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.445087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:10.446356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.449212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:10.454141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.454958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1056/2118] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:10.455947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.456165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c0b36-b9fb-4627-80d0-853155f5e3ee service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.465914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bc7f2-c2b4-431c-a943-47e9e918ac1c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.469167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1063/2119] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:10.470565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.470894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.471180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.472884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.474098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.474383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.474685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.475686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.477323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.490342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:10.491870 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.492125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1057/2120] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:10.493100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.503620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1064/2121] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:10.503779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1486bfc-7f23-474b-8ce7-2ee34910cdb4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.504733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.508837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.509502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.509785 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.510062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.510789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.512623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.514574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f05beb-a875-4326-9038-bc757cf9b8d3 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.520884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.521626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.521986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.522344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.523182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:10.523766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.523874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1058/2122] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:10.525461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.527652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.538812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcf861-d78d-49a2-87fb-5fff2e5f7c95 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.540622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1065/2123] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:10.545807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.548188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a17cd6-dc8f-4490-9122-f1a0394c5fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.549080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a17cd6-dc8f-4490-9122-f1a0394c5fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.550226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a17cd6-dc8f-4490-9122-f1a0394c5fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.550226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a17cd6-dc8f-4490-9122-f1a0394c5fbe admin admin] PATH_INFO: `/v3/projects/29af24a533de4ea38b8ad8ed5586dbf1/users/1eb1b236bac7480b9d728973e89583ef/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.559055 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb4de6-dd77-4986-9b38-70b7908d60c3 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.565218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.566080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.566685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.567113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.568556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.570637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.572108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a17cd6-dc8f-4490-9122-f1a0394c5fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=29af24a533de4ea38b8ad8ed5586dbf1, user_id=1eb1b236bac7480b9d728973e89583ef, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.575564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a17cd6-dc8f-4490-9122-f1a0394c5fbe admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.592390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1059/2124] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:10 2022] PUT /identity/v3/projects/29af24a533de4ea38b8ad8ed5586dbf1/users/1eb1b236bac7480b9d728973e89583ef/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:10.602700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.603027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.603127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1066/2125] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:10.603315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.611034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.632954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] MFA Rules not processed for user `1eb1b236bac7480b9d728973e89583ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:10.634282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:10.681094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6b958-83e9-466f-9e07-733962e89aa0 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.687298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.688051 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.688340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.688619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.689315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.691373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.712111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:10.720030 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1067/2126] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:10.721163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.721751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1060/2127] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:10.722632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.734686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b68e2-eb6c-43e0-9d78-b79ee30e2674 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.737704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc973bf7-7951-4390-b750-d113367a5dbe None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.741220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.742040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.742445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.742755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.743052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.743608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.743724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.743994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.744266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.744994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.746372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.746587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.757797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1061/2128] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:10.762426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.774706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4e2e-8f50-4abb-a5b5-ef5a87bd6403 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.779156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1068/2129] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:10.780099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.853099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce3544-5d45-41ba-9392-eeb5a81d51f7 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.859683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.860574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.861107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.861657 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.862504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.864494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.865300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.866050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.866168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.866513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.867354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.869117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.887918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1062/2130] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:10.892644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.898882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1069/2131] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:10.903345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.908394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e80a5-c2e3-4c9b-95f5-696b3335a421 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.915623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017b337-0216-4114-b91d-09aa380f2f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.917521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017b337-0216-4114-b91d-09aa380f2f9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.918011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017b337-0216-4114-b91d-09aa380f2f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.918507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017b337-0216-4114-b91d-09aa380f2f9e admin admin] PATH_INFO: `/v3/projects/09f6d100eeb94faca9d27c1d257bf642/users/b4ac8f64fadf4733a573e583d6673e66/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.926363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87435b25-c06c-42ae-88e9-bba150fac484 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.941291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:10.942108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.943039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.944070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:10.945499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.949623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.951457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017b337-0216-4114-b91d-09aa380f2f9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f6d100eeb94faca9d27c1d257bf642, user_id=b4ac8f64fadf4733a573e583d6673e66, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:10.953057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:10.954508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017b337-0216-4114-b91d-09aa380f2f9e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:10.970042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1070/2132] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:10.971044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:10.986391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1063/2133] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:10 2022] PUT /identity/v3/projects/09f6d100eeb94faca9d27c1d257bf642/users/b4ac8f64fadf4733a573e583d6673e66/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:10.986607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad8ed1-2af4-4a3b-be8b-1892285a72ee admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:10.991451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48795023-a327-4718-9720-146922dbfbf4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:10.991826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48795023-a327-4718-9720-146922dbfbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:10.991995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48795023-a327-4718-9720-146922dbfbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.029085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-48795023-a327-4718-9720-146922dbfbf4 None None] MFA Rules not processed for user `b4ac8f64fadf4733a573e583d6673e66`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:11.030899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-48795023-a327-4718-9720-146922dbfbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.101210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.102316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.102603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.102906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.103923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:11.105537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:11.120412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1071/2134] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:11.120412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.121954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.128514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1064/2135] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:11.129603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48795023-a327-4718-9720-146922dbfbf4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.148370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48795023-a327-4718-9720-146922dbfbf4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.148586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed98a8-8dbe-4a23-97ca-3f86f4042227 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.156765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.156765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.156765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.157085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.157194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] PATH_INFO: `/v3/projects/212d4e50ef064dbf9074a81bc705b186/users/8d6df76b357545a0a69032c9697a703d/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.157279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.157279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.157279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] PATH_INFO: `/v3/users/a3dd7d98f171470fb6e90e5ed199bdda` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.158495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3dd7d98f171470fb6e90e5ed199bdda)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:11.160887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:11.183005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212d4e50ef064dbf9074a81bc705b186, user_id=8d6df76b357545a0a69032c9697a703d, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:11.185664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:11.207716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1065/2136] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:11 2022] PUT /identity/v3/projects/212d4e50ef064dbf9074a81bc705b186/users/8d6df76b357545a0a69032c9697a703d/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:11.211735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.218648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3dd7d98f171470fb6e90e5ed199bdda'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.219945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3dd7d98f171470fb6e90e5ed199bdda'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.220476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.228058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3dd7d98f171470fb6e90e5ed199bdda'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.228784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3dd7d98f171470fb6e90e5ed199bdda'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.267928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:11.268508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3dd7d98f171470fb6e90e5ed199bdda'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.298515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1072/2137] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:11 2022] DELETE /identity/v3/users/a3dd7d98f171470fb6e90e5ed199bdda => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:11.302992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.307901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.310689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f144ed-4e91-4fef-b04e-1561e4c3bac2 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.385743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.386500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.386872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.387224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.389812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.470304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c190bd-b470-4d83-89c1-44e99e641c22 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.477301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.478813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.479194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.479964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] PATH_INFO: `/v3/users/87fd9305e1194ef19708230f3ad107f9` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.480833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87fd9305e1194ef19708230f3ad107f9)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:11.482862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:11.491110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:11.508587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:11.510052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db876e32-aa26-4795-8ac7-76f83a4ccc86 service glance] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:11.522237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1066/2138] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:11.534595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd3f5c-ec7d-4c64-ac57-a952be0beea6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.536701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd3f5c-ec7d-4c64-ac57-a952be0beea6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.536701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd3f5c-ec7d-4c64-ac57-a952be0beea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.545616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87fd9305e1194ef19708230f3ad107f9'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.546656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87fd9305e1194ef19708230f3ad107f9'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.558700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87fd9305e1194ef19708230f3ad107f9'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.559529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87fd9305e1194ef19708230f3ad107f9'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.577909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-38135db2-d79e-4404-8af8-b68e8fd793a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87fd9305e1194ef19708230f3ad107f9'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:11.578870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-13fd3f5c-ec7d-4c64-ac57-a952be0beea6 None None] MFA Rules not processed for user `8d6df76b357545a0a69032c9697a703d`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:11.581464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-13fd3f5c-ec7d-4c64-ac57-a952be0beea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.600194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1073/2139] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:11 2022] DELETE /identity/v3/users/87fd9305e1194ef19708230f3ad107f9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:11.601608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.602182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.602496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.632938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:11.634180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.735169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1067/2140] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:11.736549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.736851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.737135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.752796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1074/2141] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:11.755690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.762988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.769205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:11.771624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.780624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1068/2142] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:11.780624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.780624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.790187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23499c8-1c53-4c9a-b561-72f74e26c9ad None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.795647 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.796398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.796872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.797261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.797944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:11.800090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:11.831813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:11.838524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1075/2143] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:11.839925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:11.842713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:11.883113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459fd37-165a-4734-830b-5dc95208ccd9 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.954563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137c25f-d729-47e7-b462-bfa4313c2791 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:11.961564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.962297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.962638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.962985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:11.989025 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:11.990256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:11.990548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:11.990897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.007980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:12.078477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.080178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.091127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1069/2144] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:12.092045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.102746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe096c5-b591-4fd6-949e-dd11f9dfad25 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.107375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.108681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.109433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.109553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.109853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.110378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.111186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.113198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.146675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1070/2145] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:12 2022] POST /identity/v3/users => generated 299 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:12.147941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1076/2146] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:12.148375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.161543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269e24a-d7ee-4d33-9939-8fc726ffd5cc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.168170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.169125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.169926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.170373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.171752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.174542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.191247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1071/2147] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:12.195768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.210735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9c9e7-b2c6-4969-8fb6-06527250f08c service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.227603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.227603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.242144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.243771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] PATH_INFO: `/v3/projects/dc63d3d466eb48698ed467016ea33611/users/d828f9abb94049fa8b534dc459b96ce3/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.299916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63d3d466eb48698ed467016ea33611, user_id=d828f9abb94049fa8b534dc459b96ce3, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.307100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.339381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1077/2148] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:12 2022] PUT /identity/v3/projects/dc63d3d466eb48698ed467016ea33611/users/d828f9abb94049fa8b534dc459b96ce3/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:12.343352 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.361466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00132f1f-25a4-4ea7-bfe4-3be76a0e03b6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.548493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:12.570451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb8c94-15b2-4cfa-b01f-1d18fb96612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.572626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8c94-15b2-4cfa-b01f-1d18fb96612e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.574535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8c94-15b2-4cfa-b01f-1d18fb96612e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.574660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8c94-15b2-4cfa-b01f-1d18fb96612e admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.576902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb8c94-15b2-4cfa-b01f-1d18fb96612e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.579603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb8c94-15b2-4cfa-b01f-1d18fb96612e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.601613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1072/2149] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:12.605971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.624756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a035dbb-4115-434a-8eb5-46ff4a6dd16d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.637329 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.641264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.641941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.642717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] PATH_INFO: `/v3/projects/dc63d3d466eb48698ed467016ea33611/users/d828f9abb94049fa8b534dc459b96ce3/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.651121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc63d3d466eb48698ed467016ea33611, user_id=d828f9abb94049fa8b534dc459b96ce3, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.653760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.685909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1078/2150] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:12 2022] PUT /identity/v3/projects/dc63d3d466eb48698ed467016ea33611/users/d828f9abb94049fa8b534dc459b96ce3/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:12.690640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.691042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.691527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.733644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] MFA Rules not processed for user `d828f9abb94049fa8b534dc459b96ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:12.735043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:12.774256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.776816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:12.864888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1073/2151] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:12.880140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:12.883438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:12.909822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836c0d8-f869-4ca8-9257-96ececc12534 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.928871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.930180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.930682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.931234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] PATH_INFO: `/v3/projects/2b990df3465f420da8f806da92600d82` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:12.932737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b990df3465f420da8f806da92600d82)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:12.934900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:12.987545 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5e025-b615-46a2-b415-7bb349ea54db None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:12.992176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:12.992950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:12.993325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:12.993755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.004419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:13.006106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:13.030963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1074/2152] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:13.033394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:13.033984 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:13.034661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b990df3465f420da8f806da92600d82', 'request_id': 'req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.045357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b990df3465f420da8f806da92600d82', 'request_id': 'req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.058427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa8995-5b20-4196-a732-961b033f7d5b service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:13.060736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b990df3465f420da8f806da92600d82', 'request_id': 'req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.085420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] The token cache is being invalidate because project 2b990df3465f420da8f806da92600d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:13.085420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.085420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.085420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1079/2153] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:12 2022] DELETE /identity/v3/projects/2b990df3465f420da8f806da92600d82 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:13.088448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:13.090342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:13.102718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:13.151069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:13.152029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:13.152315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:13.152632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] PATH_INFO: `/v3/projects/7b326a70421a4b33b8cd5dfda7bcad69` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.154852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b326a70421a4b33b8cd5dfda7bcad69)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:13.157102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:13.203312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:13.203522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68e9a0-55fe-4d05-b7b0-5c58e405f780 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:13.211134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:13.212560 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:13.214872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:13.214872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] PATH_INFO: `/v3/users/56f904b584cf470a8560289cbaa75096` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.217174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f904b584cf470a8560289cbaa75096)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:13.220540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:13.251047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b326a70421a4b33b8cd5dfda7bcad69', 'request_id': 'req-ae79b2dc-b843-41c6-80d1-6e135952066e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.263452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b326a70421a4b33b8cd5dfda7bcad69', 'request_id': 'req-ae79b2dc-b843-41c6-80d1-6e135952066e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.292163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b326a70421a4b33b8cd5dfda7bcad69', 'request_id': 'req-ae79b2dc-b843-41c6-80d1-6e135952066e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.311284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] The token cache is being invalidate because project 7b326a70421a4b33b8cd5dfda7bcad69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:13.311617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.312314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-ae79b2dc-b843-41c6-80d1-6e135952066e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.313081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f904b584cf470a8560289cbaa75096'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.317464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1075/2154] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:13 2022] DELETE /identity/v3/projects/7b326a70421a4b33b8cd5dfda7bcad69 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:13.318912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f904b584cf470a8560289cbaa75096'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.319900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:13.320282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:13.320695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.328951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f904b584cf470a8560289cbaa75096'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.330100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f904b584cf470a8560289cbaa75096'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.347004 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:13.349708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:13.364683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9b623e01-2390-4445-914d-71ce70b5cee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f904b584cf470a8560289cbaa75096'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:13.395739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1080/2155] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:13 2022] DELETE /identity/v3/users/56f904b584cf470a8560289cbaa75096 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:13.480140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1076/2156] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:13.482671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:13.483381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:13.483893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.519640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:13.525232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:13.671949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1081/2157] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:13.675245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:13.690401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec85320-2987-403e-8b31-e24cae759b43 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:13.701440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:13.702117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:13.702400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:13.702675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.703456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:13.703769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:13.705618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:13.732963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1077/2158] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:19:13.736511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:13.746613 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3425bf-edf2-418e-a8fa-03fee5870c2c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:13.758133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:13.758817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:13.759159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:13.759405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:13.760111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:13.762020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:13.843263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1082/2159] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:13 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:13.847526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:13.865950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bde7b-b460-47a8-a96a-d438009ab7a5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:13.961050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:13.962814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:13.972751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:14.046738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.047449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.047736 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.048010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.048722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.050658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.084104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67939e18-8dde-4d76-824f-54021827a66c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.091163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.092832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.093904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.094330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] PATH_INFO: `/v3/projects/bba8fdd2698f418b8c3c2640ed2f07e3` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.096564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba8fdd2698f418b8c3c2640ed2f07e3)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.100668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.107762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1078/2160] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:13 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:14.107762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.109621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:14.173831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:14.225197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba8fdd2698f418b8c3c2640ed2f07e3', 'request_id': 'req-b9ce6aaf-6513-4850-9952-fd7073cd512f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:14.231926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e6752-431e-4a94-ae2a-b904fd6d867f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.245637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba8fdd2698f418b8c3c2640ed2f07e3', 'request_id': 'req-b9ce6aaf-6513-4850-9952-fd7073cd512f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:14.258538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba8fdd2698f418b8c3c2640ed2f07e3', 'request_id': 'req-b9ce6aaf-6513-4850-9952-fd7073cd512f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:14.288640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] The token cache is being invalidate because project bba8fdd2698f418b8c3c2640ed2f07e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:14.289378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:14.292192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:14.301150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1083/2161] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:13 2022] DELETE /identity/v3/projects/bba8fdd2698f418b8c3c2640ed2f07e3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:14.302227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.303857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:14.349000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.349947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.350410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.350782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.354926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:14.441639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce6aaf-6513-4850-9952-fd7073cd512f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.449709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2246cf7c-7b1a-4e43-a2b4-e7a25003dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.450397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246cf7c-7b1a-4e43-a2b4-e7a25003dab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.450685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246cf7c-7b1a-4e43-a2b4-e7a25003dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.450998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2246cf7c-7b1a-4e43-a2b4-e7a25003dab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.451748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246cf7c-7b1a-4e43-a2b4-e7a25003dab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.453566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2246cf7c-7b1a-4e43-a2b4-e7a25003dab8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.473470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1084/2162] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:14.475773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d0eba-f472-422e-b5ff-bcd85ed890f7 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.476068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d0eba-f472-422e-b5ff-bcd85ed890f7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.476355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d0eba-f472-422e-b5ff-bcd85ed890f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.490754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.492438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.509190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1079/2163] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:14.519167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a02d0eba-f472-422e-b5ff-bcd85ed890f7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:14.519753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.521092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a02d0eba-f472-422e-b5ff-bcd85ed890f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:14.530320 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1085/2164] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:14.540697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.540994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.541268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.543928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5904e17-5cd1-4c9e-96c3-4245343f3ade service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.554567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.555500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.555893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.556283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.557101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.560991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.582791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1080/2165] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:14.583628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:14.583628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:14.585078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.600790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fad67-c876-4b8a-a1e4-4854413b043c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.605180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1086/2166] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:14.609597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.613128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6caf8-291f-4eef-b0a8-00be98e5da3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.614242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6caf8-291f-4eef-b0a8-00be98e5da3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.615198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6caf8-291f-4eef-b0a8-00be98e5da3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.616959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6caf8-291f-4eef-b0a8-00be98e5da3e admin admin] PATH_INFO: `/v3/projects/1c2ba7c3180540f6a5f0ef6804f1f3d0/users/2198cf3f76434ff8a2fe97ff1458ae31/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.623767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547aa8-7849-412f-9c60-66413c817210 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.628743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.629417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.629697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.630651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.631487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:14.631734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.633360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.657243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6caf8-291f-4eef-b0a8-00be98e5da3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2ba7c3180540f6a5f0ef6804f1f3d0, user_id=2198cf3f76434ff8a2fe97ff1458ae31, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.657880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1087/2167] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:14.658798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.667300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6caf8-291f-4eef-b0a8-00be98e5da3e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.667491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:14.683699 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930801ad-f846-4327-a1bd-39d274af0ff4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.710237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1081/2168] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:14 2022] PUT /identity/v3/projects/1c2ba7c3180540f6a5f0ef6804f1f3d0/users/2198cf3f76434ff8a2fe97ff1458ae31/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:14.712588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.712731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.713041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.713247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.713528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.713633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.713798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.714487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.716434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.777980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1088/2169] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:14 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:14.778175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] MFA Rules not processed for user `2198cf3f76434ff8a2fe97ff1458ae31`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:14.778893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.780620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:14.913773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:14.937200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401c08b-c2f7-4741-9e07-8f2091c5fbd3 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.941574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1082/2170] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:14.943220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:14.964492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.966575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.966893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.967224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.970700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35553b8-486f-4830-ab0d-8fd295645e18 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:14.973050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.980548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:14.983360 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:14.984048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:14.984386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:14.984665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:14.985468 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:14.987708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.036041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1089/2171] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:14 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:15.036041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.060200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d773a-5304-42db-a6aa-a70571527c9d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.070443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1083/2172] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:14 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:15.071548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.076136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.076928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.077281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.077627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.079210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.086543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.110886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1090/2173] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:15.117770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.140734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbaa730-d3a2-4a49-8644-e74924a4ae7d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.253016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.253701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.253987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.254252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.255089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.256662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.261701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ade5e-4369-4966-b941-e108bb5f6b7d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.271262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.271949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.272242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.272521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.276080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1091/2174] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:15.277576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.280394 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.283790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.293347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676a085-1d75-48c3-932f-86a2fd91a59f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.298205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732c8d0-e9ec-4aa6-afb7-3d54a8c5b013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.299800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732c8d0-e9ec-4aa6-afb7-3d54a8c5b013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.300026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732c8d0-e9ec-4aa6-afb7-3d54a8c5b013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.300436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732c8d0-e9ec-4aa6-afb7-3d54a8c5b013 admin admin] PATH_INFO: `/v3/projects/760bb44f1eb24985adea8d7ebc5fafe9/users/416f32dfe42e4832b98c451bd99986d8/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.324059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732c8d0-e9ec-4aa6-afb7-3d54a8c5b013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=760bb44f1eb24985adea8d7ebc5fafe9, user_id=416f32dfe42e4832b98c451bd99986d8, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.326869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1084/2175] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:15 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:15.327617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732c8d0-e9ec-4aa6-afb7-3d54a8c5b013 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.331019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.348046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593261-49e0-46af-ac6d-8420e6e2ea05 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.358054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1092/2176] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:15 2022] PUT /identity/v3/projects/760bb44f1eb24985adea8d7ebc5fafe9/users/416f32dfe42e4832b98c451bd99986d8/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:15.359616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.359909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.360186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.393905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] MFA Rules not processed for user `416f32dfe42e4832b98c451bd99986d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:15.395170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:15.433985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:15.455628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.458585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.459091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.459601 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.460573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.462540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.483128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1085/2177] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:15.485543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.499946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db431f-2e49-4082-ab9b-d2907bbbeb33 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.509136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1093/2178] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:15.529596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.530496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.530620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.530998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.531504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.532707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:15.534437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.537319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.551197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1086/2179] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:15.572315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.597322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9c91d-557c-4262-a6eb-f75f4e32d3a6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.617718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920d77c-959c-4060-9176-c8b2b670495c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.618629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920d77c-959c-4060-9176-c8b2b670495c admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.618949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920d77c-959c-4060-9176-c8b2b670495c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.619228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920d77c-959c-4060-9176-c8b2b670495c admin admin] PATH_INFO: `/v3/projects/ebaee7be21d6471ab66130419743be62/users/2f57164f193f40afb074e6312979c72f/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.637896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483d25a-b573-4c89-b2a6-30e9f851b1f2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.640585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920d77c-959c-4060-9176-c8b2b670495c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebaee7be21d6471ab66130419743be62, user_id=2f57164f193f40afb074e6312979c72f, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.642164 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.643727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920d77c-959c-4060-9176-c8b2b670495c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.644223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.644497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.644774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.662800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.664301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.668978 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1087/2180] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:15 2022] PUT /identity/v3/projects/ebaee7be21d6471ab66130419743be62/users/2f57164f193f40afb074e6312979c72f/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:15.679523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.680064 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.680865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.692882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1094/2181] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:15.728460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.742295 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] MFA Rules not processed for user `2f57164f193f40afb074e6312979c72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:15.743589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:15.760722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42377c27-e948-41f2-9f42-bdb392982df6 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.761655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:15.890559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1088/2182] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:15.896106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.898255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.899197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.899472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.899748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] PATH_INFO: `/v3/users/416f32dfe42e4832b98c451bd99986d8` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.902080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] RBAC: Authorizing `identity:delete_user(user_id=416f32dfe42e4832b98c451bd99986d8)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.904670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.918961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fdb8b-ab7e-4a55-b335-4e1a4c0ed907 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:15.927270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:15.927947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:15.928241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:15.928525 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:15.929344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:15.935427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:15.985441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1089/2183] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:15 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:15.988648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:15.990219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416f32dfe42e4832b98c451bd99986d8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:15.991279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416f32dfe42e4832b98c451bd99986d8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:16.007458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416f32dfe42e4832b98c451bd99986d8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:16.010696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416f32dfe42e4832b98c451bd99986d8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:16.058470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:16.079998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f3e54263-cfd6-4780-a6e5-8a50c021438a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416f32dfe42e4832b98c451bd99986d8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:16.103731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1095/2184] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:15 2022] DELETE /identity/v3/users/416f32dfe42e4832b98c451bd99986d8 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:16.108058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.108934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.108934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.119511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc2f26-6bc0-4146-a47f-72fa788bfab6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:16.141633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:16.143338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:16.246492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:16.247422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.247875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.248164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.248906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:16.250997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:16.267538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1096/2185] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:16.284748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1090/2186] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:15 2022] POST /identity/v3/users => generated 298 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:16.289687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.289858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:16.296055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.297268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:16.309819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db406e60-defa-4588-b7a7-a9d1ab99c6bc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:16.314799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:16.315522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.315818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.316098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.316979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:16.318908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:16.334774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1091/2187] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:16.339470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.356452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64385ccf-1f26-459a-afb9-7da58718696e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:16.367297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:16.368611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.368784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.369013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] PATH_INFO: `/v3/projects/64c3a839a0114501b2aa22de108915fd/users/956f6d74be5f46ca9a9891f70988f37e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.392326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f79bb0-b580-4656-8d61-a9aec300507d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:16.397416 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:16.398290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.398722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.399177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.413274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:16.414913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:16.434686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:16.440735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1097/2188] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:16.444148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c3a839a0114501b2aa22de108915fd, user_id=956f6d74be5f46ca9a9891f70988f37e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:16.448577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:16.475776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1092/2189] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:16 2022] PUT /identity/v3/projects/64c3a839a0114501b2aa22de108915fd/users/956f6d74be5f46ca9a9891f70988f37e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:16.480241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.492874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543a044-d960-4e1c-a486-f37c1b8512c3 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:16.570479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad5e3d-66c1-4eb3-a7af-87bb55c8f01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:16.571414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5e3d-66c1-4eb3-a7af-87bb55c8f01d admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.571778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5e3d-66c1-4eb3-a7af-87bb55c8f01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.572160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad5e3d-66c1-4eb3-a7af-87bb55c8f01d admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.576764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad5e3d-66c1-4eb3-a7af-87bb55c8f01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:16.578570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad5e3d-66c1-4eb3-a7af-87bb55c8f01d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:16.598771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1098/2190] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:16.601908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.620011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bac54b-de5c-40a7-a017-e2db439d2280 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:16.631390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:16.632301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.632589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.632928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] PATH_INFO: `/v3/projects/64c3a839a0114501b2aa22de108915fd/users/956f6d74be5f46ca9a9891f70988f37e/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.639622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c3a839a0114501b2aa22de108915fd, user_id=956f6d74be5f46ca9a9891f70988f37e, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:16.652480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:16.681581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1093/2191] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:16 2022] PUT /identity/v3/projects/64c3a839a0114501b2aa22de108915fd/users/956f6d74be5f46ca9a9891f70988f37e/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:16.693668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:16.693999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:16.694283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:16.752310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] MFA Rules not processed for user `956f6d74be5f46ca9a9891f70988f37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:16.755270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:16.775999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:16.869758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1099/2192] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:16.888064 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.890249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:16.955694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:16.957663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:17.015300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362fc80-62b1-4f91-b429-5bc3d0f0656e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.021180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.022030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.022414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.022784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.026460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:17.073245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b604c9-fad7-46a7-bc34-eb50bc84ec8a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.081838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.083967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.083967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.084598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] PATH_INFO: `/v3/projects/760bb44f1eb24985adea8d7ebc5fafe9` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.086564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=760bb44f1eb24985adea8d7ebc5fafe9)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.090725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.123700 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.125917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb6fa-a0bd-4fdb-85ef-13d6e31c08f2 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.140939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1094/2193] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:17.157685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760bb44f1eb24985adea8d7ebc5fafe9', 'request_id': 'req-09a66204-0f19-4a51-8b80-b501310a2bfb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:17.167561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760bb44f1eb24985adea8d7ebc5fafe9', 'request_id': 'req-09a66204-0f19-4a51-8b80-b501310a2bfb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:17.175309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760bb44f1eb24985adea8d7ebc5fafe9', 'request_id': 'req-09a66204-0f19-4a51-8b80-b501310a2bfb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:17.188332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] The token cache is being invalidate because project 760bb44f1eb24985adea8d7ebc5fafe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:17.188738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:17.189687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09a66204-0f19-4a51-8b80-b501310a2bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:17.196206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1100/2194] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:16 2022] DELETE /identity/v3/projects/760bb44f1eb24985adea8d7ebc5fafe9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:17.197244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c90bd-b35e-4485-abf1-012a75f468f4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.197547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c90bd-b35e-4485-abf1-012a75f468f4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.197825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c90bd-b35e-4485-abf1-012a75f468f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.222138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-807c90bd-b35e-4485-abf1-012a75f468f4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:17.223606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-807c90bd-b35e-4485-abf1-012a75f468f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:17.244162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.244871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.245408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.279331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] MFA Rules not processed for user `326e0d70244f46059664a135aefab0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:17.280973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:17.309591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1095/2195] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:17.319696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.320112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.320572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.325559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:17.342479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:17.343829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:17.353277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1096/2196] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:17.357522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.367302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb39b4a-4416-49f3-8c95-7afa83064b49 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.373006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.373775 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.374083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.374373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.375289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1101/2197] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:17.375483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:17.375741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.377398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.392296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.393699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1097/2198] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:17.394027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:17.398789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.413273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f1cd-1cb2-452c-a5df-c7a67cd95504 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.437508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.438195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.438480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.438783 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.439605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.443987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.459150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:17.482339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308ae8c-b79b-4953-9aaa-7a8fda231c1a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.492233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.496525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.497071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.497189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.499032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1098/2199] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:17 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:17.500127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.508359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.518514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.526148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1102/2200] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:17.625085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c3511-0774-4b85-8d39-437ad4b3aed5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.632074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.632790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.633054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.633463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.634001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.639792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.665356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1099/2201] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:17 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:17.668717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.682361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04272d-4eeb-47af-a67b-fcb96ab1dfb0 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.688228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.689000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.689286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.689580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.690393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.692025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.705150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1103/2202] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:17.708848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.720458 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d5e30-4664-48fb-a9a0-024779d70c43 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.728338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65aeb8-abdc-4506-8381-f5026f27e1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.728971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65aeb8-abdc-4506-8381-f5026f27e1ca admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.729265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65aeb8-abdc-4506-8381-f5026f27e1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.729588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65aeb8-abdc-4506-8381-f5026f27e1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.730629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65aeb8-abdc-4506-8381-f5026f27e1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.733167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65aeb8-abdc-4506-8381-f5026f27e1ca admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.748743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1100/2203] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:17.752673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:17.786011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1fa0b-7f89-4096-9f54-cebc6ed12ac0 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:17.797391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:17.797391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.797650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.797921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] PATH_INFO: `/v3/projects/332e3c4417b245d58c8b14e7c4a163e4/users/9679a579ad8d41c08d7dd2d83505c7f7/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.874467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=332e3c4417b245d58c8b14e7c4a163e4, user_id=9679a579ad8d41c08d7dd2d83505c7f7, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:17.878722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:17.907543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1104/2204] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:17 2022] PUT /identity/v3/projects/332e3c4417b245d58c8b14e7c4a163e4/users/9679a579ad8d41c08d7dd2d83505c7f7/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:17.918676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:17.921682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:17.921960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:17.984417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] MFA Rules not processed for user `9679a579ad8d41c08d7dd2d83505c7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:17.985903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:18.085729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1101/2205] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:18.097686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:18.100013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:18.186292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ae48b-7fe6-42a5-87a0-cc36eeb6f2fa admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:18.191664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:18.192985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:18.193377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:18.197903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:18.217152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:18.218792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:18.228696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1105/2206] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:18.876477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:18.878293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:18.970014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3deaa76-4b64-44e6-a25e-6a6841d21fb3 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:18.974137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:18.975120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:18.975641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:18.976174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:18.978522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:18.996337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:19.067998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:19.069598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129aff5-b9db-497f-acc8-254b880ce405 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:19.082668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1102/2207] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:19.319307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:19.321470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:19.358988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bccbc2-9bba-4e35-815e-9a2115656752 service glance] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:19.365534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:19.367106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:19.367592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:19.368028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:19.370396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:19.419013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:19.500896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:19.502514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:19.542326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1106/2208] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:19.911180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:19.911664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:19.911781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:19.948699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] MFA Rules not processed for user `326e0d70244f46059664a135aefab0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:19.949972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:20.045172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1103/2209] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:20.125271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:20.134557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14592dcf-6d11-4f17-9153-f6b5461aa80d service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:20.139632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:20.140287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:20.140569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:20.140844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:20.143571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:20.229958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:20.231909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91307205-951d-45e8-96bd-bad73ea0a5e4 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:20.243356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1107/2210] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:22.031809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:22.033445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:22.109844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5802089-4484-4d56-b248-0dc0712a6a29 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:22.117190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:22.118157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:22.118514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:22.118914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] PATH_INFO: `/v3/users/1eb1b236bac7480b9d728973e89583ef` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:22.120301 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb1b236bac7480b9d728973e89583ef)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:22.122546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:22.170070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb1b236bac7480b9d728973e89583ef'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:22.170718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb1b236bac7480b9d728973e89583ef'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:22.178294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb1b236bac7480b9d728973e89583ef'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:22.178988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb1b236bac7480b9d728973e89583ef'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:22.189830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:22.195442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb1b236bac7480b9d728973e89583ef'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:22.215784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1104/2211] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:22 2022] DELETE /identity/v3/users/1eb1b236bac7480b9d728973e89583ef => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:22.220678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:22.221179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:22.221262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:22.242013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:22.243281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:22.303643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1108/2212] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:22.313086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:22.314502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:22.376384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51dee90-e1b6-4475-b81d-a17cd6ed7d5b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:22.380656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:22.381298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:22.381585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:22.381860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:22.390901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:22.392332 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:22.401683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1105/2213] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:22.908795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:22.910353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:22.923081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d6bfa-88d2-4c16-9e05-d0662006097b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:22.929092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:22.930208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:22.930674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:22.931173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] PATH_INFO: `/v3/projects/29af24a533de4ea38b8ad8ed5586dbf1` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:22.933290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=29af24a533de4ea38b8ad8ed5586dbf1)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:22.936325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:22.999167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29af24a533de4ea38b8ad8ed5586dbf1', 'request_id': 'req-09206f63-4f50-45ca-a2cf-38413f2fc0eb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.000931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:23.008905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29af24a533de4ea38b8ad8ed5586dbf1', 'request_id': 'req-09206f63-4f50-45ca-a2cf-38413f2fc0eb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.015565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29af24a533de4ea38b8ad8ed5586dbf1', 'request_id': 'req-09206f63-4f50-45ca-a2cf-38413f2fc0eb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.027520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] The token cache is being invalidate because project 29af24a533de4ea38b8ad8ed5586dbf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:23.027919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.028747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.030463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1109/2214] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:22 2022] DELETE /identity/v3/projects/29af24a533de4ea38b8ad8ed5586dbf1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:23.034523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:23.045630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1e36-3b88-49f8-9c46-703e9790d034 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:23.144407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:23.145226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:23.145660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:23.145830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] PATH_INFO: `/v3/users/8d6df76b357545a0a69032c9697a703d` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:23.147203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6df76b357545a0a69032c9697a703d)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:23.149146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:23.214631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6df76b357545a0a69032c9697a703d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.215323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6df76b357545a0a69032c9697a703d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.223109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6df76b357545a0a69032c9697a703d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.224338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6df76b357545a0a69032c9697a703d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.231659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6df76b357545a0a69032c9697a703d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.244931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:23.249450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1106/2215] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:23 2022] DELETE /identity/v3/users/8d6df76b357545a0a69032c9697a703d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:23.527544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:23.529057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:23.550469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09206f63-4f50-45ca-a2cf-38413f2fc0eb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:23.609682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:23.610660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:23.610987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:23.611268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] PATH_INFO: `/v3/projects/212d4e50ef064dbf9074a81bc705b186` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:23.612737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212d4e50ef064dbf9074a81bc705b186)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:23.614707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:23.659572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212d4e50ef064dbf9074a81bc705b186', 'request_id': 'req-37e4466d-15f7-44a3-bc87-350514ddc847'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.667811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212d4e50ef064dbf9074a81bc705b186', 'request_id': 'req-37e4466d-15f7-44a3-bc87-350514ddc847'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.673439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212d4e50ef064dbf9074a81bc705b186', 'request_id': 'req-37e4466d-15f7-44a3-bc87-350514ddc847'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.685288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] The token cache is being invalidate because project 212d4e50ef064dbf9074a81bc705b186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:23.685577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.686196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.687788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1110/2216] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:23 2022] DELETE /identity/v3/projects/212d4e50ef064dbf9074a81bc705b186 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:23.691725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:23.694056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:23.707194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca11b7-bc5f-48e0-ba22-dc74dbb2a4b7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:23.767546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:23.768509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:23.768798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:23.769126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] PATH_INFO: `/v3/users/d828f9abb94049fa8b534dc459b96ce3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:23.770317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d828f9abb94049fa8b534dc459b96ce3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:23.772315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:23.821248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd828f9abb94049fa8b534dc459b96ce3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.821906 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd828f9abb94049fa8b534dc459b96ce3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.828870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd828f9abb94049fa8b534dc459b96ce3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.831994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd828f9abb94049fa8b534dc459b96ce3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.841773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd828f9abb94049fa8b534dc459b96ce3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:23.861590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1107/2217] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:23 2022] DELETE /identity/v3/users/d828f9abb94049fa8b534dc459b96ce3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:24.136499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.138177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.156482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4466d-15f7-44a3-bc87-350514ddc847 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.218246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.225992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.259375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.260370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.260660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.260936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] PATH_INFO: `/v3/projects/dc63d3d466eb48698ed467016ea33611` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.263622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc63d3d466eb48698ed467016ea33611)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.265739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.279080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:24.314176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994657e-6a1d-4f11-bc7e-d70ca2a0d67e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.323056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc63d3d466eb48698ed467016ea33611', 'request_id': 'req-d146d03d-a657-4ce0-82ac-f0720cf6278b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.335424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc63d3d466eb48698ed467016ea33611', 'request_id': 'req-d146d03d-a657-4ce0-82ac-f0720cf6278b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.345199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc63d3d466eb48698ed467016ea33611', 'request_id': 'req-d146d03d-a657-4ce0-82ac-f0720cf6278b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.345793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:24.365643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] The token cache is being invalidate because project dc63d3d466eb48698ed467016ea33611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:24.366138 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.366668 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d146d03d-a657-4ce0-82ac-f0720cf6278b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.368321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1111/2218] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:24 2022] DELETE /identity/v3/projects/dc63d3d466eb48698ed467016ea33611 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:24.374006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad01c2c-791b-4e03-a136-726f8cbc744f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.374472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad01c2c-791b-4e03-a136-726f8cbc744f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.374745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad01c2c-791b-4e03-a136-726f8cbc744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.392057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.393039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.393416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.393907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] PATH_INFO: `/v3/users/2198cf3f76434ff8a2fe97ff1458ae31` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.395237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2198cf3f76434ff8a2fe97ff1458ae31)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.396934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9ad01c2c-791b-4e03-a136-726f8cbc744f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:24.398077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.398200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9ad01c2c-791b-4e03-a136-726f8cbc744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.407924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1112/2219] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:24.425475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.425475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.425475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.438708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:24.439946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.450877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1113/2220] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:24.454646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.466754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430447df-8db0-4060-9039-ae3132e9aa22 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.471724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.472639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.472925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.473200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.474475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:24.474778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.476415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.477565 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2198cf3f76434ff8a2fe97ff1458ae31'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.478612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2198cf3f76434ff8a2fe97ff1458ae31'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.485986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2198cf3f76434ff8a2fe97ff1458ae31'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.486759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2198cf3f76434ff8a2fe97ff1458ae31'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.489498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1114/2221] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:19:24.494107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.495840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c7c5bdf5-c8e1-426b-b228-bbd14b9210fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2198cf3f76434ff8a2fe97ff1458ae31'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:24.495995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.508472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e5ffc-ed9a-46ec-a771-1057e682ad8e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.513303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.513985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.514288 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.514713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.515464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.515733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1108/2222] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:24 2022] DELETE /identity/v3/users/2198cf3f76434ff8a2fe97ff1458ae31 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:24.517485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.519909 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.520195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.520991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.540582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:24.540582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.545293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1115/2223] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:24 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:24.549296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.605912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1109/2224] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:24.614273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f014992-488a-441e-8c41-97445a1f1b28 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.615923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.617280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.619377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.620056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.620341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.620620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.621321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.623303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.648047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1116/2225] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:24 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:24.649369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.660663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588866-3d6f-4f16-b959-b02baf1f8828 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.667214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.668017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.668305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.668580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.669330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.670938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.675400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:24.681120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1117/2226] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:24.682331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb582e-b7df-4be2-810d-a0bcf4f7697c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.685332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.695497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d312ce6-b880-4b5a-8afa-0e53fce02933 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.700504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.701158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.701439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.701711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.702540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.704470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.719144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1118/2227] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:24.720123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.730523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ad7b8-3ea7-42ab-9aad-fc00d7d3fc77 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.735154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.736006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.736287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.736411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0bce4-2bff-41cc-9fea-12c6803d0028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.736564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] PATH_INFO: `/v3/projects/d1aac41f4f074858ba560987a158c38b/users/51d5d87476eb4c8d8ceacf022fe4135d/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.737062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0bce4-2bff-41cc-9fea-12c6803d0028 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.737351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0bce4-2bff-41cc-9fea-12c6803d0028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.737631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0bce4-2bff-41cc-9fea-12c6803d0028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.750114 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0bce4-2bff-41cc-9fea-12c6803d0028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.751703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0bce4-2bff-41cc-9fea-12c6803d0028 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.756948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1aac41f4f074858ba560987a158c38b, user_id=51d5d87476eb4c8d8ceacf022fe4135d, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.759882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.762269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:24.762388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1110/2228] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:24.776086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1119/2229] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:24 2022] PUT /identity/v3/projects/d1aac41f4f074858ba560987a158c38b/users/51d5d87476eb4c8d8ceacf022fe4135d/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:24.781186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.781551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.781893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.799748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:24.800746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.864804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1111/2230] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:24.874756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:24.876530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:24.944867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1f47-7584-45b1-9173-ffb52f51ca1e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:24.950315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:24.951118 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:24.951498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:24.951846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:24.963516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:24.965036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:24.975829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1120/2231] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:25.124723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:25.126773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:25.139495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc491cb-aaf3-4c68-8609-efb5e92b1ad6 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:25.145192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:25.146298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:25.146682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:25.147048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] PATH_INFO: `/v3/projects/1c2ba7c3180540f6a5f0ef6804f1f3d0` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:25.148653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2ba7c3180540f6a5f0ef6804f1f3d0)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:25.150731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:25.208478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2ba7c3180540f6a5f0ef6804f1f3d0', 'request_id': 'req-3f8de111-6539-4815-b282-f60a2134a053'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.216527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2ba7c3180540f6a5f0ef6804f1f3d0', 'request_id': 'req-3f8de111-6539-4815-b282-f60a2134a053'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.225126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2ba7c3180540f6a5f0ef6804f1f3d0', 'request_id': 'req-3f8de111-6539-4815-b282-f60a2134a053'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.237685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] The token cache is being invalidate because project 1c2ba7c3180540f6a5f0ef6804f1f3d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:25.237991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.238690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.240771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1112/2232] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:25 2022] DELETE /identity/v3/projects/1c2ba7c3180540f6a5f0ef6804f1f3d0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:25.244877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:25.255649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c2c63-be3b-4d5a-9c85-04a4249422ab service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:25.345818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:25.346784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:25.347414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:25.347751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] PATH_INFO: `/v3/users/2f57164f193f40afb074e6312979c72f` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:25.349046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f57164f193f40afb074e6312979c72f)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:25.351082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:25.408012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f57164f193f40afb074e6312979c72f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.408733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f57164f193f40afb074e6312979c72f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.416024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f57164f193f40afb074e6312979c72f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.417133 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f57164f193f40afb074e6312979c72f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.452271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1a17a11a-4db3-4a54-98c8-c9aae6eeff43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f57164f193f40afb074e6312979c72f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:25.472681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1121/2233] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:25 2022] DELETE /identity/v3/users/2f57164f193f40afb074e6312979c72f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:25.929646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:25.931147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:25.948768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:25.951920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:25.952238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:25.967400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8de111-6539-4815-b282-f60a2134a053 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:25.988079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] MFA Rules not processed for user `51d5d87476eb4c8d8ceacf022fe4135d`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:25.989948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:25.994216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:26.008617 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:26.051195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:26.052821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.053103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.053376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] PATH_INFO: `/v3/projects/ebaee7be21d6471ab66130419743be62` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.054622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebaee7be21d6471ab66130419743be62)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:26.056813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:26.069192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1122/2234] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:26.081923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:26.083775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:26.111355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebaee7be21d6471ab66130419743be62', 'request_id': 'req-33ec10a6-e3ca-4181-851d-b39be18ad6b2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.118574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebaee7be21d6471ab66130419743be62', 'request_id': 'req-33ec10a6-e3ca-4181-851d-b39be18ad6b2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.126649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebaee7be21d6471ab66130419743be62', 'request_id': 'req-33ec10a6-e3ca-4181-851d-b39be18ad6b2'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.138726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] The token cache is being invalidate because project ebaee7be21d6471ab66130419743be62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:26.139065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.139752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.141697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1113/2235] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:25 2022] DELETE /identity/v3/projects/ebaee7be21d6471ab66130419743be62 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:26.145451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:26.146822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:26.148338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae2b58-ecd6-4ff5-b3ea-68c9a5bada96 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:26.201535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:26.202112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.202417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.202676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.204742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:26.213880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec10a6-e3ca-4181-851d-b39be18ad6b2 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:26.218628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:26.219526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.219806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.220081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] PATH_INFO: `/v3/users/956f6d74be5f46ca9a9891f70988f37e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.221291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=956f6d74be5f46ca9a9891f70988f37e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:26.223480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:26.275425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:26.277337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956f6d74be5f46ca9a9891f70988f37e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.277882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:26.278341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956f6d74be5f46ca9a9891f70988f37e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.288224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1123/2236] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:19:26.288412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956f6d74be5f46ca9a9891f70988f37e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.288996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956f6d74be5f46ca9a9891f70988f37e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.293917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:26.298023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-291d374c-e2c1-48a9-94c6-173eb17345e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956f6d74be5f46ca9a9891f70988f37e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.320380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1114/2237] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:26 2022] DELETE /identity/v3/users/956f6d74be5f46ca9a9891f70988f37e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:19:26.643017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:26.661895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72128b-a108-4cbf-8d1d-020a060d5f79 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:26.685990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:26.737483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:26.738348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.738652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.738979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] PATH_INFO: `/v3/projects/64c3a839a0114501b2aa22de108915fd` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.740702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c3a839a0114501b2aa22de108915fd)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:26.742619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:26.789622 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c3a839a0114501b2aa22de108915fd', 'request_id': 'req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.796729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c3a839a0114501b2aa22de108915fd', 'request_id': 'req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.803586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c3a839a0114501b2aa22de108915fd', 'request_id': 'req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.815028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] The token cache is being invalidate because project 64c3a839a0114501b2aa22de108915fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:26.815381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.816027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-35f64b2c-92c0-4f4f-88f5-7eabeb59919b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:26.818636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1124/2238] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:26 2022] DELETE /identity/v3/projects/64c3a839a0114501b2aa22de108915fd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:26.823652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.823942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.824218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.842011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:26.843298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:26.903387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1115/2239] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:26.912117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.912460 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.912749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.931006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:26.932289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:26.940290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1125/2240] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:26.944529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:26.954447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66d957-3286-4938-9b9f-afbbb3673d1a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:26.960355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:26.961166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.961511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.961854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.962806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:26.963155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:26.965090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:26.977752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1116/2241] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:19:26.981978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:26.991509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ec1d3-0c72-463a-aded-64209eb75913 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:26.997054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:26.997809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:26.998236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:26.998541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:26.999328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:27.001250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:27.029059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1126/2242] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:26 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:27.033234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:27.044036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495370f-3034-42fd-a6d9-ab777cfb89f9 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:27.101313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:27.101982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.102266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.102539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.103291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:27.105214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:27.125327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1117/2243] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:27 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:27.129405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:27.140144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580ae97-6bfd-4a5e-8e6c-c3338c364827 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:27.144878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:27.145541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.145826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.146104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.146858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:27.148470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:27.159132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1127/2244] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:19:27.163650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:27.173823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc89c23-f42b-4453-91e0-2fde47f1d94b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:27.179050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ef7dd-3db7-41b4-ad7a-c944e4f9b55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:27.179708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef7dd-3db7-41b4-ad7a-c944e4f9b55d admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.179992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef7dd-3db7-41b4-ad7a-c944e4f9b55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.180270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef7dd-3db7-41b4-ad7a-c944e4f9b55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.180994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ef7dd-3db7-41b4-ad7a-c944e4f9b55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:27.182630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ef7dd-3db7-41b4-ad7a-c944e4f9b55d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:27.192929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1118/2245] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:27.197292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:27.211392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5ac63-007f-4b92-962d-2acc0632f2c3 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:27.217392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:27.218502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.218785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.219246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] PATH_INFO: `/v3/projects/8c38e45e60ff4f5595a4b3bec32db245/users/e15c2a65a3434df5abf5dc4ce53fddc2/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.235652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c38e45e60ff4f5595a4b3bec32db245, user_id=e15c2a65a3434df5abf5dc4ce53fddc2, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:27.238306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:27.252461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1128/2246] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:27 2022] PUT /identity/v3/projects/8c38e45e60ff4f5595a4b3bec32db245/users/e15c2a65a3434df5abf5dc4ce53fddc2/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:27.321932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17cd9a-1cc8-4e09-8a88-2d709108fcb4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.322507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17cd9a-1cc8-4e09-8a88-2d709108fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.322507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17cd9a-1cc8-4e09-8a88-2d709108fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.349764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-dc17cd9a-1cc8-4e09-8a88-2d709108fcb4 None None] MFA Rules not processed for user `e15c2a65a3434df5abf5dc4ce53fddc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:27.351027 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-dc17cd9a-1cc8-4e09-8a88-2d709108fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:27.386052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:27.429286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1119/2247] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:27.439094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:27.440840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:27.487840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:27.501458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e4ca4-91c5-4e4b-8806-a594dc794fed admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:27.505759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6904c-589c-474e-b8a2-0f067bce8eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:27.506368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6904c-589c-474e-b8a2-0f067bce8eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.506653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6904c-589c-474e-b8a2-0f067bce8eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.506964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6904c-589c-474e-b8a2-0f067bce8eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.515619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6904c-589c-474e-b8a2-0f067bce8eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:27.517059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6904c-589c-474e-b8a2-0f067bce8eb9 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:27.526020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1129/2248] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:27.689220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:27.689512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:27.689789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:27.708967 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] MFA Rules not processed for user `51d5d87476eb4c8d8ceacf022fe4135d`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:27.710185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:27.778447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1120/2249] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:28.891919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2de72-6a30-41e6-87ea-aca5de2ef615 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:28.892508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2de72-6a30-41e6-87ea-aca5de2ef615 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:28.892508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2de72-6a30-41e6-87ea-aca5de2ef615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:28.910229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-bea2de72-6a30-41e6-87ea-aca5de2ef615 None None] MFA Rules not processed for user `e15c2a65a3434df5abf5dc4ce53fddc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:28.911500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bea2de72-6a30-41e6-87ea-aca5de2ef615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:28.918427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1130/2250] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:37.509312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:37.511019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:37.599116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd6f6e-3ba6-4dea-a88b-1b5bb9fe3376 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:37.604164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:37.605139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:37.605425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:37.605703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] PATH_INFO: `/v3/users/b4ac8f64fadf4733a573e583d6673e66` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:37.607529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ac8f64fadf4733a573e583d6673e66)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:37.610080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:37.672943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ac8f64fadf4733a573e583d6673e66'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:37.674547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ac8f64fadf4733a573e583d6673e66'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:37.685037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ac8f64fadf4733a573e583d6673e66'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:37.686052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ac8f64fadf4733a573e583d6673e66'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:37.697106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ac8f64fadf4733a573e583d6673e66'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:37.715814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:37.726915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1121/2251] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:19:37 2022] DELETE /identity/v3/users/b4ac8f64fadf4733a573e583d6673e66 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:37.730238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:37.730728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:37.730810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:37.754636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:37.756074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:37.851385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1131/2252] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:37.858813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:37.860379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:37.948441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa495c06-0c9f-4d50-802f-83c4a6e039c1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:37.953408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5ac64-2cef-473a-862e-743abc8a1367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:37.960412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5ac64-2cef-473a-862e-743abc8a1367 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:37.960887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5ac64-2cef-473a-862e-743abc8a1367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:37.961274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5ac64-2cef-473a-862e-743abc8a1367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:37.973162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5ac64-2cef-473a-862e-743abc8a1367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:37.975688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5ac64-2cef-473a-862e-743abc8a1367 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:37.993266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1122/2253] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:38.425598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.427783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:38.446902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e42bf8-d72d-4232-b1c2-b09f5487351a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.452453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.453863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.454273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.455897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] PATH_INFO: `/v3/projects/09f6d100eeb94faca9d27c1d257bf642` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.457671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f6d100eeb94faca9d27c1d257bf642)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.459937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.512765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f6d100eeb94faca9d27c1d257bf642', 'request_id': 'req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:38.519987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f6d100eeb94faca9d27c1d257bf642', 'request_id': 'req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:38.528029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f6d100eeb94faca9d27c1d257bf642', 'request_id': 'req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:38.541330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] The token cache is being invalidate because project 09f6d100eeb94faca9d27c1d257bf642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:19:38.541618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:38.542269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a39b1e8a-813e-4eeb-9e32-b9b0b296062a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:19:38.543828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1132/2254] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:19:38 2022] DELETE /identity/v3/projects/09f6d100eeb94faca9d27c1d257bf642 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:38.548983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.549455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.549558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.567657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:38.568934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:38.627815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1123/2255] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:38.635192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.635503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.635805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.654735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:38.655962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:38.664327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1133/2256] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:19:38.668517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.678987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cae1e-5afc-4e02-b401-326c26d5fb48 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.683735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.684398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.684681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.684956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.685718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:19:38.686028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.687694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.699307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1124/2257] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:19:38.703529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.713366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b444b17-d74a-4a4a-8dcd-fc86dceca4ec None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.717935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.718576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.718882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.719158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.719878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.721770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.748248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1134/2258] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:19:38 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:19:38.752268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.762151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb082ed3-b282-4938-b507-5c2a851c10ab admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.817462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.818186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.818416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.818696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.819430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.821383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.844098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1125/2259] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:19:38 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:19:38.845269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.856694 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289467e6-fc83-4bb8-ae46-8585fbb62424 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.862342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.863086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.863373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.863650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.864452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.866386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.877309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1135/2260] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:38.881222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.902939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13ca2c-b5cb-4eae-a7f2-c36c9683379e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.907741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd6e54-ff08-4263-9528-7d7238636bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.908406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd6e54-ff08-4263-9528-7d7238636bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.908691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd6e54-ff08-4263-9528-7d7238636bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.908966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd6e54-ff08-4263-9528-7d7238636bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.909694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd6e54-ff08-4263-9528-7d7238636bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.912029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd6e54-ff08-4263-9528-7d7238636bd9 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.925394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1126/2261] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:19:38.926676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:38.937455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa9ebc-1659-450c-93b5-8327de0c1f7e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:38.942165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:38.943113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.943396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.943671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] PATH_INFO: `/v3/projects/1304267434194936ad7c457961cab231/users/d668e551d8a34d6fbb4bb9a399248eac/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:38.959002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1304267434194936ad7c457961cab231, user_id=d668e551d8a34d6fbb4bb9a399248eac, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:38.961663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:38.971389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:38.986924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1136/2262] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:19:38 2022] PUT /identity/v3/projects/1304267434194936ad7c457961cab231/users/d668e551d8a34d6fbb4bb9a399248eac/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:19:38.989353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:38.989649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:38.989932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:39.002975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:39.017723 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] MFA Rules not processed for user `d668e551d8a34d6fbb4bb9a399248eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:19:39.018968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:39.104767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1127/2263] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:19:39.118681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:19:39.120802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:19:39.176527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf76332-119c-46f1-8238-5d8318e2964a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:19:39.180735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9709fb6-45b4-4a2f-b2af-08cbb7470186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:19:39.181371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9709fb6-45b4-4a2f-b2af-08cbb7470186 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:19:39.181650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9709fb6-45b4-4a2f-b2af-08cbb7470186 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:19:39.182549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9709fb6-45b4-4a2f-b2af-08cbb7470186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:19:39.190282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9709fb6-45b4-4a2f-b2af-08cbb7470186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:19:39.191736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9709fb6-45b4-4a2f-b2af-08cbb7470186 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:19:39.200630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1137/2264] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:19:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:19:49.107392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=29 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:19:49.196494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=27 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:07.432887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:07.581825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce946b58-58fb-42b0-a4df-ddb1b3708743 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:07.586557 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:07.587455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:07.587741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:07.588035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] PATH_INFO: `/v3/users/d668e551d8a34d6fbb4bb9a399248eac` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:07.589772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d668e551d8a34d6fbb4bb9a399248eac)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:07.591752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:07.641467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd668e551d8a34d6fbb4bb9a399248eac'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:07.642127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd668e551d8a34d6fbb4bb9a399248eac'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:07.649365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd668e551d8a34d6fbb4bb9a399248eac'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:07.650023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd668e551d8a34d6fbb4bb9a399248eac'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:07.658337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd668e551d8a34d6fbb4bb9a399248eac'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:07.675482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1128/2265] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:07 2022] DELETE /identity/v3/users/d668e551d8a34d6fbb4bb9a399248eac => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:07.679871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:07.680413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:07.680517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:07.727600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:07.728827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:07.787389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1138/2266] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:07.796670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:07.798072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:07.839846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:07.895479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47233d86-66d2-4061-8264-51aa50ff8e8b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:07.904787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba22bfe-0ae2-437f-aa26-144efeb8e5ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:07.905462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22bfe-0ae2-437f-aa26-144efeb8e5ee service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:07.905745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22bfe-0ae2-437f-aa26-144efeb8e5ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:07.906022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba22bfe-0ae2-437f-aa26-144efeb8e5ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:07.916731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba22bfe-0ae2-437f-aa26-144efeb8e5ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:07.918204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba22bfe-0ae2-437f-aa26-144efeb8e5ee service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:07.931590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1129/2267] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:08.387377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:08.389723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:08.426729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488539-3f15-4702-a7a6-29ebf261582f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:08.444798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:08.447527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.447867 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.448147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] PATH_INFO: `/v3/projects/1304267434194936ad7c457961cab231` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.449452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1304267434194936ad7c457961cab231)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:08.451502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:08.512255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1304267434194936ad7c457961cab231', 'request_id': 'req-04d9d097-04bf-45a3-a609-090d4d3aa71f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:08.522498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1304267434194936ad7c457961cab231', 'request_id': 'req-04d9d097-04bf-45a3-a609-090d4d3aa71f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:08.529413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1304267434194936ad7c457961cab231', 'request_id': 'req-04d9d097-04bf-45a3-a609-090d4d3aa71f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:08.551049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] The token cache is being invalidate because project 1304267434194936ad7c457961cab231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:08.551933 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:08.552862 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-04d9d097-04bf-45a3-a609-090d4d3aa71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:08.554362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1139/2268] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:08 2022] DELETE /identity/v3/projects/1304267434194936ad7c457961cab231 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:08.559711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aa093-df7b-417c-bc4c-6c62ad6353f9 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.560130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aa093-df7b-417c-bc4c-6c62ad6353f9 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.560270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8aa093-df7b-417c-bc4c-6c62ad6353f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.612714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3e8aa093-df7b-417c-bc4c-6c62ad6353f9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:08.614093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3e8aa093-df7b-417c-bc4c-6c62ad6353f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:08.696029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1130/2269] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:08.704702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.705665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.706105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.708421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.709382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.709382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.727307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] MFA Rules not processed for user `51d5d87476eb4c8d8ceacf022fe4135d`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:08.728606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:08.735359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:08.736550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:08.747571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1140/2270] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:08.751901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:08.761930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d630262-7a00-4178-a6f7-7bbb2166507b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:08.767365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:08.768017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.768303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.768579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.769307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:20:08.769607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:08.771387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:08.784193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1141/2271] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:20:08.788405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:08.798323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e9d76-34f7-408f-8975-376b1e180e97 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:08.803203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1131/2272] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:08.803323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:08.803704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.803986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.804260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.804941 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:08.807650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:08.840380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1142/2273] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:08 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:20:08.841979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:08.857144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd696316-2b51-49d1-96f6-6b93ba086afd None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:08.894919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:08.924101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:08.924790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.925075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.925354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.926643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:08.928741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:08.963370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1132/2274] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:08 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:08.964581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:08.980078 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93dc49-839d-414f-961c-9ebb4ae28611 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:08.988636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:08.989841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:08.990130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:08.990405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:08.991195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:08.997340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:08.999389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:09.013915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1143/2275] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:09.017871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.028763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81085e50-10c5-4d25-ae35-1a0d99548d27 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.035267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04a9de-79fe-49d0-b659-d70ca5af39cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.036282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04a9de-79fe-49d0-b659-d70ca5af39cd admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.036382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04a9de-79fe-49d0-b659-d70ca5af39cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.036608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04a9de-79fe-49d0-b659-d70ca5af39cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.037348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04a9de-79fe-49d0-b659-d70ca5af39cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.038983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04a9de-79fe-49d0-b659-d70ca5af39cd admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.057093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.063711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1133/2276] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:09.068025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f82bc-0a36-4e71-aaa6-271bd2c46678 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.096356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.097677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.102489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.103301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] PATH_INFO: `/v3/projects/7b6d1c12f8b54839bc0d8d204dc1e76b/users/77c3aa20ccda4c02a00de23522c7bca4/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.145363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6d1c12f8b54839bc0d8d204dc1e76b, user_id=77c3aa20ccda4c02a00de23522c7bca4, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.151885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.173701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1144/2277] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:09 2022] PUT /identity/v3/projects/7b6d1c12f8b54839bc0d8d204dc1e76b/users/77c3aa20ccda4c02a00de23522c7bca4/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:09.185021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.185354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.185638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.241402 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] MFA Rules not processed for user `77c3aa20ccda4c02a00de23522c7bca4`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:09.242799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:09.351649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1134/2278] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:09.356086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.422442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fff5c9-8607-4e57-8c7e-45c4036a675a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.428740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.429974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.430262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.430536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.431266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.433742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.464266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1145/2279] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:09 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:09.468363 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.483178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca9c13-24e2-4ac7-ada7-7945dc2c00e6 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.561064 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.561990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.562420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.562705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.563453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.565384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.594966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.595110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1135/2280] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:09 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:09.611435 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fe9e3-a76c-4661-906a-ff2f4bae1f86 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.616297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.617017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.617307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.617586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.618437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.621586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.632469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1146/2281] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:20:09.636875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.647696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df701-9a50-4edb-8555-495e8e01865a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.652694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f6910-b727-4c71-81a2-60a1cffb80c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.653130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f6910-b727-4c71-81a2-60a1cffb80c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.653414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f6910-b727-4c71-81a2-60a1cffb80c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.653690 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f6910-b727-4c71-81a2-60a1cffb80c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.654423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f6910-b727-4c71-81a2-60a1cffb80c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.660364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f6910-b727-4c71-81a2-60a1cffb80c3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.674691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1136/2282] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:09.675763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.687236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640c99c-9e8b-4873-b081-3cf23eef1a01 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.692818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.693226 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.693509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.693815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] PATH_INFO: `/v3/projects/9131aabed8a84eb8abba8378e03052f4/users/29079fe88d9a48fdb7c3863bae749ecd/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.712038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9131aabed8a84eb8abba8378e03052f4, user_id=29079fe88d9a48fdb7c3863bae749ecd, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.715399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.735926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1147/2283] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:09 2022] PUT /identity/v3/projects/9131aabed8a84eb8abba8378e03052f4/users/29079fe88d9a48fdb7c3863bae749ecd/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:09.746305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.746589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.747098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.790444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] MFA Rules not processed for user `29079fe88d9a48fdb7c3863bae749ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:09.792224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:09.803051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:09.880674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1137/2284] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:09.881627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.944841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf87807-90df-432c-b814-f6aa8872ba6c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:09.950083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:09.950733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:09.951048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:09.951370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:09.952061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:09.953990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:09.978616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1148/2285] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:09 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:09.983322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:09.994222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603e7a5-d6d6-4364-beae-0d9809266331 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.051502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.052193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.052484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.052761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.053451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.055387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.075457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1138/2286] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:09 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:10.079559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.090652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0eb787-f5c6-46f4-8482-36cc8137a1b2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.095338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.095997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.096280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.096557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.097285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.099177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.100409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:10.112205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1149/2287] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:10.116030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.125920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa4539-6bab-4012-8459-221ca54da2cd admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.130491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.131392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.131675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.131953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] PATH_INFO: `/v3/projects/7af9d61632aa4773b7b16a151add567b/users/6563d9a8bd51467d83403f806ac88786/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.147237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af9d61632aa4773b7b16a151add567b, user_id=6563d9a8bd51467d83403f806ac88786, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.149908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.164673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1139/2288] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:10 2022] PUT /identity/v3/projects/7af9d61632aa4773b7b16a151add567b/users/6563d9a8bd51467d83403f806ac88786/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:10.169202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.180191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f7827-ea82-4650-89c9-5ddce5ffd6b4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.242600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba2570-1e5b-402e-99de-a99db0f26581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.243334 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba2570-1e5b-402e-99de-a99db0f26581 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.243634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba2570-1e5b-402e-99de-a99db0f26581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.243930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba2570-1e5b-402e-99de-a99db0f26581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.244685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba2570-1e5b-402e-99de-a99db0f26581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.246277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba2570-1e5b-402e-99de-a99db0f26581 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.255945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1150/2289] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:10.260113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.270341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364c249-c6b9-463d-be0e-422786705633 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.275165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.276579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.276859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.277145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] PATH_INFO: `/v3/projects/7af9d61632aa4773b7b16a151add567b/users/6563d9a8bd51467d83403f806ac88786/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.279686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af9d61632aa4773b7b16a151add567b, user_id=6563d9a8bd51467d83403f806ac88786, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.282541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.296489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1140/2290] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:10 2022] PUT /identity/v3/projects/7af9d61632aa4773b7b16a151add567b/users/6563d9a8bd51467d83403f806ac88786/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:10.306577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.307057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.307196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.308566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.310327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:10.336295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] MFA Rules not processed for user `6563d9a8bd51467d83403f806ac88786`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:10.337514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:10.385548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b3bad-f3c9-4061-8800-1680f1450573 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.391919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.393002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.393306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.393556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] PATH_INFO: `/v3/users/9679a579ad8d41c08d7dd2d83505c7f7` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.394885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9679a579ad8d41c08d7dd2d83505c7f7)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.396839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.402467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:10.412068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1151/2291] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:10.423947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.425534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:10.452831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9679a579ad8d41c08d7dd2d83505c7f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:10.453640 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9679a579ad8d41c08d7dd2d83505c7f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:10.461902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9679a579ad8d41c08d7dd2d83505c7f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:10.462571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9679a579ad8d41c08d7dd2d83505c7f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:10.471487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e051546d-21e9-4691-bcbc-79614969a354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9679a579ad8d41c08d7dd2d83505c7f7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:10.484525 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce5fae0-a908-4e7e-ad57-8159a8e19ecc None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.488219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.488859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.489142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.489818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1141/2292] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:10 2022] DELETE /identity/v3/users/9679a579ad8d41c08d7dd2d83505c7f7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:10.490080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.492023 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:10.494053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.494449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.494722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.511377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:10.512554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:10.525201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:10.556352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.557922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.567864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1152/2293] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:20:10.575890 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1142/2294] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:10.585476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:10.587758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:10.689304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bad040-b716-475d-ae09-7b68a2abdb1c service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:10.744945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ef97b-b1f9-47d8-8020-999aa0fb5a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:10.745582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ef97b-b1f9-47d8-8020-999aa0fb5a6e service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:10.745780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ef97b-b1f9-47d8-8020-999aa0fb5a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:10.746079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ef97b-b1f9-47d8-8020-999aa0fb5a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:10.755545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ef97b-b1f9-47d8-8020-999aa0fb5a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:10.757106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ef97b-b1f9-47d8-8020-999aa0fb5a6e service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:10.773995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1153/2295] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:11.236950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.238628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:11.253506 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db7233-6ae2-4e69-b27e-4cb30093c25e None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.258671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.260024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.260024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.260248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] PATH_INFO: `/v3/projects/332e3c4417b245d58c8b14e7c4a163e4` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.261439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=332e3c4417b245d58c8b14e7c4a163e4)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.263493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.323460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332e3c4417b245d58c8b14e7c4a163e4', 'request_id': 'req-a9bf60ef-8016-487d-b126-c7da2804bc02'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:11.330507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332e3c4417b245d58c8b14e7c4a163e4', 'request_id': 'req-a9bf60ef-8016-487d-b126-c7da2804bc02'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:11.355426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332e3c4417b245d58c8b14e7c4a163e4', 'request_id': 'req-a9bf60ef-8016-487d-b126-c7da2804bc02'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:11.380292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] The token cache is being invalidate because project 332e3c4417b245d58c8b14e7c4a163e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:11.382109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:11.385821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a9bf60ef-8016-487d-b126-c7da2804bc02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:11.394119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1143/2296] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:11 2022] DELETE /identity/v3/projects/332e3c4417b245d58c8b14e7c4a163e4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:11.397057 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.397629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.398106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.446160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:11.451684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:11.538607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1154/2297] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:11.547877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.548173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.548478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.561720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:11.568251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:11.570183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:11.578621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1144/2298] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:11.582882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.593267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df626c08-6c4b-4b8f-bd83-9df70038c17b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.597829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.598470 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.599034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.599310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.600035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:20:11.600335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.601956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.613973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1155/2299] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:20:11.618267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.627529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be00ca-d1d1-48a1-b6bf-c3f58fcb2cd2 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.632108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.632747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.633030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.633304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.633981 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.635915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.661593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1145/2300] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:11 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:11.665728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.676185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cab97-06b5-4e7e-a752-b19f9ba2734b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.688400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:11.731764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.732423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.732704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.732977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.733692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.735729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.759071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1156/2301] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:11 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:11.760962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.771273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13287c5-0d36-4c92-a024-0d5849ef99c4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.776399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.776991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.777380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.777537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.778257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.779945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.790881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1146/2302] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:11.795786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.805823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614a694-daa0-432f-bec3-d309c1765cc4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.811000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc66c6-542e-435d-a09b-814a62479541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.811660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc66c6-542e-435d-a09b-814a62479541 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.811944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc66c6-542e-435d-a09b-814a62479541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.812221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc66c6-542e-435d-a09b-814a62479541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.812950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc66c6-542e-435d-a09b-814a62479541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.814615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc66c6-542e-435d-a09b-814a62479541 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.828429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1157/2303] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:11.832421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:11.842741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905ea0f-e72e-4b7c-9f09-710b040230b6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:11.847263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:11.848120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.848399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.848927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] PATH_INFO: `/v3/projects/6d069f71a9d74f48b4aa79290bf6e3f1/users/5e240eb390854a1bbe3125854f4d059e/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.864069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d069f71a9d74f48b4aa79290bf6e3f1, user_id=5e240eb390854a1bbe3125854f4d059e, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:11.867003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:11.880356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1147/2304] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:11 2022] PUT /identity/v3/projects/6d069f71a9d74f48b4aa79290bf6e3f1/users/5e240eb390854a1bbe3125854f4d059e/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:11.892110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bf9a6-b06d-4165-8010-0f274258d4f5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:11.892403 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bf9a6-b06d-4165-8010-0f274258d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:11.892682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bf9a6-b06d-4165-8010-0f274258d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:11.922419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-121bf9a6-b06d-4165-8010-0f274258d4f5 None None] MFA Rules not processed for user `5e240eb390854a1bbe3125854f4d059e`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:11.924033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-121bf9a6-b06d-4165-8010-0f274258d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:12.047483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1158/2305] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:12.069282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:12.082503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:12.182042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e77ca-d0e8-48f7-afc8-1247c9c53d1f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:12.186005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:12.186671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:12.187222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:12.187511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:12.199956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:12.201395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:12.214759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1148/2306] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:12.281943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:12.282637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:12.282803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:12.308245 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:12.309725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:12.320048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:12.321245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:12.374042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e468340-650e-49ac-ad42-f7036f446d27 service glance] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:12.378333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:12.379349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:12.379641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:12.379949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:12.382527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:12.386806 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1159/2307] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:20:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:12.440890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:12.442420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:12.460471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:12.462039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:12.469450 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:12.480289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:12.492013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1149/2308] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:20:12.566871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b62a46-caa6-43af-b3bd-88c216400df7 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:12.589415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8e7d2-1f8e-41c1-8c7a-282dad650057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:12.590815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8e7d2-1f8e-41c1-8c7a-282dad650057 service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:12.590815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8e7d2-1f8e-41c1-8c7a-282dad650057 service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:12.590815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8e7d2-1f8e-41c1-8c7a-282dad650057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:12.608866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8e7d2-1f8e-41c1-8c7a-282dad650057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:12.614300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8e7d2-1f8e-41c1-8c7a-282dad650057 service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:12.634712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1160/2309] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 13:20:13.591418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:13.592899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:13.604216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd3d6d-191b-4d7a-8cea-41fdd096677f service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:13.611214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:13.611884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:13.612167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:13.612413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:13.614572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:13.684742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:13.685255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:13.685340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:13.711319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:13.713984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:13.715179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:13.717623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749f146-1cef-4fd9-a05e-edddd663d9c6 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:13.724139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1161/2310] 10.222.0.61 () {58 vars in 967 bytes} [Tue Jun 7 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:13.737906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:13.738754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1150/2311] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:13.754702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d15e4e-3cfc-454a-a9f2-a92a16be5c10 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:13.762432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:13.763225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:13.763510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:13.763784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:13.773970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:13.787721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:13.803345 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1162/2312] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:20:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 13:20:14.147772 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251f33-4be1-4343-8073-abd95267f7da None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:14.148071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251f33-4be1-4343-8073-abd95267f7da None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:14.148358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251f33-4be1-4343-8073-abd95267f7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:14.169502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e3251f33-4be1-4343-8073-abd95267f7da None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:14.170738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e3251f33-4be1-4343-8073-abd95267f7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:14.179641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1151/2313] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:14.199212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:14.213073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1812d6-79ef-4a03-b61c-72442b8ac0ab service swift] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:14.220172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:14.221215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:14.222200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:14.222778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:14.233916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:14.235674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:14.244467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1163/2314] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:20:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:20:14.613682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637412cd-a763-42e9-9d29-3cc9f19ec217 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:14.614624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637412cd-a763-42e9-9d29-3cc9f19ec217 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:14.615189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637412cd-a763-42e9-9d29-3cc9f19ec217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:14.635554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-637412cd-a763-42e9-9d29-3cc9f19ec217 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:14.636818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-637412cd-a763-42e9-9d29-3cc9f19ec217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:14.644667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1152/2315] 10.222.0.61 () {58 vars in 967 bytes} [Tue Jun 7 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:14.657063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:14.669215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c8789-cec9-4ed7-878a-9ceb5c56aeb8 service swift] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:14.675500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:14.675600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:14.675882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:14.676158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:14.685565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:14.687192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:14.696475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1164/2316] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:20:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:20:15.252843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb40b6-4cbd-42a7-9eb9-aab147b6a214 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:15.253143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb40b6-4cbd-42a7-9eb9-aab147b6a214 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:15.253610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb40b6-4cbd-42a7-9eb9-aab147b6a214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:15.288063 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-effb40b6-4cbd-42a7-9eb9-aab147b6a214 None None] MFA Rules not processed for user `49c60c4956f8479fbfe93f9f85a8d497`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:15.294454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-effb40b6-4cbd-42a7-9eb9-aab147b6a214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:15.419657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1153/2317] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:16.999627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.001104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.073929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4dee8-9baf-4bf4-896d-5b8a7bf696af service swift] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.078510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.079411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.079693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.080284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] PATH_INFO: `/v3/users/e15c2a65a3434df5abf5dc4ce53fddc2` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.081471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15c2a65a3434df5abf5dc4ce53fddc2)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.083467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.129730 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15c2a65a3434df5abf5dc4ce53fddc2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.130289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15c2a65a3434df5abf5dc4ce53fddc2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.138686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15c2a65a3434df5abf5dc4ce53fddc2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.139844 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15c2a65a3434df5abf5dc4ce53fddc2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.147948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:17.153563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15c2a65a3434df5abf5dc4ce53fddc2'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.170553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1165/2318] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:16 2022] DELETE /identity/v3/users/e15c2a65a3434df5abf5dc4ce53fddc2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:17.172067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.172511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.172655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.190536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:17.191846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.253699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1154/2319] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:17.263554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.264963 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.328302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935849ff-e19d-4ff4-a7ed-8829dd2e4748 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.332910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ca971-8d0f-46d5-b982-a37abd1438fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.333616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca971-8d0f-46d5-b982-a37abd1438fd service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.333907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca971-8d0f-46d5-b982-a37abd1438fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.334190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ca971-8d0f-46d5-b982-a37abd1438fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.344190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ca971-8d0f-46d5-b982-a37abd1438fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.345689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ca971-8d0f-46d5-b982-a37abd1438fd service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.355948 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1166/2320] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:17.533075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.534611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.548378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46260e-d0f9-48de-80aa-077e09aece3c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.553626 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.555963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.556528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.557709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] PATH_INFO: `/v3/projects/8c38e45e60ff4f5595a4b3bec32db245` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.559505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c38e45e60ff4f5595a4b3bec32db245)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.561979 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.587118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:17.610056 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c38e45e60ff4f5595a4b3bec32db245', 'request_id': 'req-e6df067d-4f95-4c85-a15c-11fc3c44c19e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.616023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c38e45e60ff4f5595a4b3bec32db245', 'request_id': 'req-e6df067d-4f95-4c85-a15c-11fc3c44c19e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.623637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c38e45e60ff4f5595a4b3bec32db245', 'request_id': 'req-e6df067d-4f95-4c85-a15c-11fc3c44c19e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.633868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] The token cache is being invalidate because project 8c38e45e60ff4f5595a4b3bec32db245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:17.634086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.634702 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-e6df067d-4f95-4c85-a15c-11fc3c44c19e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:17.636875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1155/2321] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:17 2022] DELETE /identity/v3/projects/8c38e45e60ff4f5595a4b3bec32db245 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:17.642289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.642576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.643062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.661045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:17.662391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.719995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1167/2322] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:17.727877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.728175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.728461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.746926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:17.748130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.756127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1156/2323] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:17.760091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.771734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e54973-0036-48db-8b1a-84e1121df78a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.776942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.777605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.777887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.778167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.778924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:20:17.779229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.780877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.793629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1168/2324] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:20:17.797954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.807163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2d492-be37-484b-98b3-37e273c4387c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.811859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.812499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.812784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.813057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.813737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.815699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.845153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1157/2325] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:17 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:17.846284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.857352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3414f-5f05-46d6-ad46-56ad749c81b4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.932841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.933516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.933800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.934077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.934770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.937053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.964674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1169/2326] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:17 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:17.968973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.974637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:17.976600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:17.980983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e5371-5364-4a07-9126-101c5b3f9bad admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.984831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:17.986811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.987728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.988042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.988292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.989102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:17.989700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100b7de-d42d-41c1-993b-96c3831cdb5c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:17.990714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:17.994667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:17.995593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:17.995876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:17.996151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] PATH_INFO: `/v3/users/77c3aa20ccda4c02a00de23522c7bca4` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:17.997770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c3aa20ccda4c02a00de23522c7bca4)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:18.000418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:18.006581 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1158/2327] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:18.010966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:18.025528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c39165-37b8-4127-83fa-554e46d41b11 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:18.033072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:18.034891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:18.034891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:18.034891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:18.035290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:18.036796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:18.049318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1159/2328] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:20:18.053406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:18.063918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c3aa20ccda4c02a00de23522c7bca4'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:18.064077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dae23-1903-43a6-9161-ec1849e24a64 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:18.064659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c3aa20ccda4c02a00de23522c7bca4'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:18.069046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f47b30-211a-4776-9e1a-fbbe3dfe7548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:18.069919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47b30-211a-4776-9e1a-fbbe3dfe7548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:18.070199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47b30-211a-4776-9e1a-fbbe3dfe7548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:18.070497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47b30-211a-4776-9e1a-fbbe3dfe7548 admin admin] PATH_INFO: `/v3/projects/b562a09561af4197bf2835b31a9dd34e/users/36c5458a18f146be87207e0f6dd598eb/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:18.079346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c3aa20ccda4c02a00de23522c7bca4'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:18.080366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c3aa20ccda4c02a00de23522c7bca4'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:18.087121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f47b30-211a-4776-9e1a-fbbe3dfe7548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b562a09561af4197bf2835b31a9dd34e, user_id=36c5458a18f146be87207e0f6dd598eb, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:18.090356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f47b30-211a-4776-9e1a-fbbe3dfe7548 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:18.095832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-00dd0aac-c8fb-4287-8010-5f5ee4ebb557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c3aa20ccda4c02a00de23522c7bca4'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:18.108272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1160/2329] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:18 2022] PUT /identity/v3/projects/b562a09561af4197bf2835b31a9dd34e/users/36c5458a18f146be87207e0f6dd598eb/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:20:18.110908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1170/2330] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:17 2022] DELETE /identity/v3/users/77c3aa20ccda4c02a00de23522c7bca4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:20:18.113624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:18.113983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:18.114369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:18.115146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:18.115444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:18.115814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:18.132447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:18.133774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:18.142046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:18.143335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:18.206453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1161/2331] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:18.219119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:18.220102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1171/2332] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:18.220857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:18.230011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:18.231879 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:18.274205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:18.319717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc628e-567b-440f-ab6f-7f2ee2850778 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:18.322329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4df4c-d1f7-4651-a042-87a0ed0fa9fa None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:18.324732 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:18.325406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:18.325716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:18.326200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:18.328134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:18.329137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:18.329642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:18.330109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:18.342869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:18.343737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:18.351390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:18.353164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:18.367526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1162/2333] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:18.370092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1172/2334] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:20:19.492933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:19.495138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:19.510788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a91a2-28f8-436b-9124-a24694924183 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:19.517797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:19.519715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:19.520318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:19.520793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] PATH_INFO: `/v3/projects/7b6d1c12f8b54839bc0d8d204dc1e76b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:19.532686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6d1c12f8b54839bc0d8d204dc1e76b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:19.538176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:19.629960 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6d1c12f8b54839bc0d8d204dc1e76b', 'request_id': 'req-616f55c9-cd7c-4d32-b201-01f070858ec0'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.637107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6d1c12f8b54839bc0d8d204dc1e76b', 'request_id': 'req-616f55c9-cd7c-4d32-b201-01f070858ec0'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.648905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6d1c12f8b54839bc0d8d204dc1e76b', 'request_id': 'req-616f55c9-cd7c-4d32-b201-01f070858ec0'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.664146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] The token cache is being invalidate because project 7b6d1c12f8b54839bc0d8d204dc1e76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:19.664437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.665149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-616f55c9-cd7c-4d32-b201-01f070858ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.667291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1163/2335] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:19 2022] DELETE /identity/v3/projects/7b6d1c12f8b54839bc0d8d204dc1e76b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:19.671478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:19.681370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ac2a6-ddd4-4850-be7f-dfc88335930f service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:19.702025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:19.758909 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:19.759847 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:19.760666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:19.761154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] PATH_INFO: `/v3/users/29079fe88d9a48fdb7c3863bae749ecd` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:19.762605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29079fe88d9a48fdb7c3863bae749ecd)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:19.764396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:19.830591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29079fe88d9a48fdb7c3863bae749ecd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.831274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29079fe88d9a48fdb7c3863bae749ecd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.838914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29079fe88d9a48fdb7c3863bae749ecd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.839644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29079fe88d9a48fdb7c3863bae749ecd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.857912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29079fe88d9a48fdb7c3863bae749ecd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:19.874925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1173/2336] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:19 2022] DELETE /identity/v3/users/29079fe88d9a48fdb7c3863bae749ecd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:20.134302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:20.134588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:20.134914 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:20.167858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] MFA Rules not processed for user `dcf5286e9ca14f56905e6e247e411896`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:20.170274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:20.303074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1164/2337] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:20.663754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:20.665272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:20.783913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ad7d0-cfaa-42e1-b62b-95422eaab0d7 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:20.794415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:20.795519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:20.796539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:20.796908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] PATH_INFO: `/v3/projects/9131aabed8a84eb8abba8378e03052f4` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:20.799706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9131aabed8a84eb8abba8378e03052f4)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:20.803596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:20.879069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:20.886123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9131aabed8a84eb8abba8378e03052f4', 'request_id': 'req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:20.896425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9131aabed8a84eb8abba8378e03052f4', 'request_id': 'req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:20.909318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9131aabed8a84eb8abba8378e03052f4', 'request_id': 'req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:20.931054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] The token cache is being invalidate because project 9131aabed8a84eb8abba8378e03052f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:20.931054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:20.931054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:20.938734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1174/2338] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:20 2022] DELETE /identity/v3/projects/9131aabed8a84eb8abba8378e03052f4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:20.942216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:20.957591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156bd85-5cf8-41eb-9f71-083d79c6e78b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:21.052366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:21.055166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:21.055675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:21.056174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] PATH_INFO: `/v3/users/6563d9a8bd51467d83403f806ac88786` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:21.058212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6563d9a8bd51467d83403f806ac88786)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:21.061205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:21.068780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:21.132814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6563d9a8bd51467d83403f806ac88786'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:21.133535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6563d9a8bd51467d83403f806ac88786'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:21.143774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6563d9a8bd51467d83403f806ac88786'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:21.144544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6563d9a8bd51467d83403f806ac88786'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:21.163043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5230389e-cb29-4845-9899-1c070eec4630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6563d9a8bd51467d83403f806ac88786'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:21.186128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1165/2339] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:20 2022] DELETE /identity/v3/users/6563d9a8bd51467d83403f806ac88786 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:21.733398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:21.747834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:21.792061 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9ae9-e6a1-4ffb-87e0-4281aa28d674 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:21.892716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:21.893920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:21.894422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:21.899161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] PATH_INFO: `/v3/projects/7af9d61632aa4773b7b16a151add567b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:21.901739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af9d61632aa4773b7b16a151add567b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:21.907231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:21.978960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af9d61632aa4773b7b16a151add567b', 'request_id': 'req-0a3af60c-4825-4ebe-9bfe-1c425d568a92'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:22.040488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af9d61632aa4773b7b16a151add567b', 'request_id': 'req-0a3af60c-4825-4ebe-9bfe-1c425d568a92'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:22.047817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af9d61632aa4773b7b16a151add567b', 'request_id': 'req-0a3af60c-4825-4ebe-9bfe-1c425d568a92'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:22.060808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] The token cache is being invalidate because project 7af9d61632aa4773b7b16a151add567b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:22.061383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:22.062039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0a3af60c-4825-4ebe-9bfe-1c425d568a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:22.063995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1175/2340] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:21 2022] DELETE /identity/v3/projects/7af9d61632aa4773b7b16a151add567b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:22.071298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.072451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.073758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.105945 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:22.107241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:22.263732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1166/2341] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:22.271555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.272029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.272379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.304943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:22.306572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:22.325435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:22.332347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1176/2342] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:22.342465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8a55c-ec34-4285-8497-eaf0dfff6bad None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:22.355528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:22.356215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.356502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.356780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.357539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:20:22.357848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:22.359503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:22.381264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1167/2343] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:20:22.387561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:22.407980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373e71-ccb8-41f4-b185-3d54f99363fc None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:22.433036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:22.434285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.434285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.434285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.434990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:22.442661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:22.501019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1177/2344] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:22 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:22.505141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:22.516065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7de4cc-b93a-498b-a46c-28d51a5bfa4a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:22.601794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:22.602487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.602878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.603095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.605541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:22.606361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:22.661368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1168/2345] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:22 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:22.662796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:22.678290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae941da-4991-417f-9947-91c6e8c4a173 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:22.707707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:22.709393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.709592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.710257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.711712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:22.716789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:22.743180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1178/2346] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:22.747315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:22.754618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:22.788333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91102b88-f971-4c70-bd9f-1650f63dcf3b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:22.797093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34cc2ec-607f-47f3-82de-edb054a78b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:22.798675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cc2ec-607f-47f3-82de-edb054a78b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.799144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cc2ec-607f-47f3-82de-edb054a78b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.800080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34cc2ec-607f-47f3-82de-edb054a78b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.801209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cc2ec-607f-47f3-82de-edb054a78b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:22.804203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34cc2ec-607f-47f3-82de-edb054a78b1b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:22.845107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:22.845651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1169/2347] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:22.858254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d775b8-6276-4464-bc9e-2772decd33fa admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:22.870724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:22.871679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.871961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.872239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] PATH_INFO: `/v3/projects/93ccd4ad7de64582b36c7ecb78e7bc04/users/97a107569a0b4baa9103b145313178ab/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.906058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ccd4ad7de64582b36c7ecb78e7bc04, user_id=97a107569a0b4baa9103b145313178ab, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:22.908797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:22.925431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:22.943596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1179/2348] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:22 2022] PUT /identity/v3/projects/93ccd4ad7de64582b36c7ecb78e7bc04/users/97a107569a0b4baa9103b145313178ab/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:20:22.955062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4198d-5aa3-4f43-a46d-90295f060b81 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:22.955357 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4198d-5aa3-4f43-a46d-90295f060b81 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:22.955680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4198d-5aa3-4f43-a46d-90295f060b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:22.999784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cef4198d-5aa3-4f43-a46d-90295f060b81 None None] MFA Rules not processed for user `97a107569a0b4baa9103b145313178ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:23.001593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cef4198d-5aa3-4f43-a46d-90295f060b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:23.157546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1170/2349] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:23.173692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:23.175528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:23.255904 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5745cf-3b03-46a3-bf87-03df34f7d097 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:23.263393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:23.264359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:23.264828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:23.265287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:23.276382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:23.278601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:23.293599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1180/2350] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:26.816820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:26.817476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:26.817476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:26.843455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] MFA Rules not processed for user `36c5458a18f146be87207e0f6dd598eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:26.844661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:26.910368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1171/2351] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:26.915229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:26.917385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:26.982351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be078b38-54b6-48f7-b3f1-8c9b9d976cb2 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:26.987307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:26.987975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:26.988260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:26.988565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:26.989360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:26.991316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:27.018290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1181/2352] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:26 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:27.022407 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:27.036790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81a9b7-618b-45ed-9e8b-73e09d4698e5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:27.106505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:27.107487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:27.107931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:27.108355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:27.109388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:27.111821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:27.132728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:27.139286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1172/2353] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:27 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:27.143611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:27.154876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190023-8b98-48c2-bd1d-a086e8ea5596 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:27.159584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:27.160252 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:27.160537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:27.160816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:27.161555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:27.166066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:27.174934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1182/2354] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:27.179075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:27.189528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec872a8-b9a2-448e-9858-cf0119abf9ab admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:27.194242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:27.195134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:27.195412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:27.195682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] PATH_INFO: `/v3/projects/08b3a41c875d4efa9e5dade725f4e828/users/3ed5603ee21d4999a3a94304b84dbff1/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:27.211149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b3a41c875d4efa9e5dade725f4e828, user_id=3ed5603ee21d4999a3a94304b84dbff1, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:27.214463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:27.229186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1173/2355] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:27 2022] PUT /identity/v3/projects/08b3a41c875d4efa9e5dade725f4e828/users/3ed5603ee21d4999a3a94304b84dbff1/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:20:27.233095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:27.243041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee77bb-69d6-4b35-a249-9967e4913f4e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:27.301773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:27.302440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:27.302724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:27.303104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:27.303821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:27.305441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:27.316898 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1183/2356] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:27.320996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:27.332445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fba2e-d499-462c-90e6-f5d2c3d7d41f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:27.337482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d875552-5470-4d4b-be44-0b7c86f4cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:27.338379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875552-5470-4d4b-be44-0b7c86f4cdab admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:27.338671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875552-5470-4d4b-be44-0b7c86f4cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:27.338983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d875552-5470-4d4b-be44-0b7c86f4cdab admin admin] PATH_INFO: `/v3/projects/08b3a41c875d4efa9e5dade725f4e828/users/3ed5603ee21d4999a3a94304b84dbff1/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:27.341065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d875552-5470-4d4b-be44-0b7c86f4cdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b3a41c875d4efa9e5dade725f4e828, user_id=3ed5603ee21d4999a3a94304b84dbff1, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:27.343762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d875552-5470-4d4b-be44-0b7c86f4cdab admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:27.354757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1174/2357] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:27 2022] PUT /identity/v3/projects/08b3a41c875d4efa9e5dade725f4e828/users/3ed5603ee21d4999a3a94304b84dbff1/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:33.214915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:33.225936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a432-20d8-4a32-8b2a-c7eb1633d0b5 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:33.236758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:33.310861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:33.311791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:33.312574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:33.313042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] PATH_INFO: `/v3/users/97a107569a0b4baa9103b145313178ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:33.315989 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a107569a0b4baa9103b145313178ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:33.318939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:33.403986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a107569a0b4baa9103b145313178ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:33.404624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a107569a0b4baa9103b145313178ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:33.415237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a107569a0b4baa9103b145313178ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:33.416125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a107569a0b4baa9103b145313178ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:33.438623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-73a93952-8dd1-4a09-8578-3a70cf8475ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a107569a0b4baa9103b145313178ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:33.459738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1184/2358] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:33 2022] DELETE /identity/v3/users/97a107569a0b4baa9103b145313178ab => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:33.467229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d9ff3-1549-459d-917e-09173a4699da None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:33.467733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d9ff3-1549-459d-917e-09173a4699da None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:33.467818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d9ff3-1549-459d-917e-09173a4699da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:33.489223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-409d9ff3-1549-459d-917e-09173a4699da None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:33.491055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-409d9ff3-1549-459d-917e-09173a4699da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:33.510644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:33.511563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:33.511892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:33.544360 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] MFA Rules not processed for user `3ed5603ee21d4999a3a94304b84dbff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:33.545869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:33.578997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1175/2359] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:33.591651 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d9ff3-1549-459d-917e-09173a4699da None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:33.595952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-409d9ff3-1549-459d-917e-09173a4699da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:33.675254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1185/2360] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:20:33.683604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d9ff3-1549-459d-917e-09173a4699da None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:33.690177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:33.690861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:33.691172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:33.691449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:33.701633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:33.703105 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:33.713277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1176/2361] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:33.929400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:33.933808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:33.958063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:34.081128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8009279-c436-4f65-b237-42c6070a2159 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:34.090274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:34.091351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:34.091522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:34.091755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:34.093601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:34.202677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:34.204412 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78002e-709e-4e94-a17c-f5864f6c6b31 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:34.216081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1186/2362] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:34.234649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:34.238237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:34.258051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8f5f8-9323-4150-bc61-38266d48d419 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:34.274530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:34.276100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:34.282552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:34.284292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] PATH_INFO: `/v3/projects/93ccd4ad7de64582b36c7ecb78e7bc04` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:34.288322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ccd4ad7de64582b36c7ecb78e7bc04)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:34.291296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:34.388561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ccd4ad7de64582b36c7ecb78e7bc04', 'request_id': 'req-4ca6807d-1ad7-4e51-9c98-6695016e06c3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:34.401648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ccd4ad7de64582b36c7ecb78e7bc04', 'request_id': 'req-4ca6807d-1ad7-4e51-9c98-6695016e06c3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:34.431379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ccd4ad7de64582b36c7ecb78e7bc04', 'request_id': 'req-4ca6807d-1ad7-4e51-9c98-6695016e06c3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:34.436342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:34.453818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] The token cache is being invalidate because project 93ccd4ad7de64582b36c7ecb78e7bc04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:34.454120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:34.454971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:34.458368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1177/2363] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:34 2022] DELETE /identity/v3/projects/93ccd4ad7de64582b36c7ecb78e7bc04 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:44.216283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=62 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:44.456650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=2,keystone:SELECT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:46.874281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:46.874792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:46.874916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:46.924928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:46.926147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:46.991823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1187/2364] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:20:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:47.004065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:47.005465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:47.103251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca6807d-1ad7-4e51-9c98-6695016e06c3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:47.109125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d695-2a55-414b-80ec-9f74b39118e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:47.109778 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d695-2a55-414b-80ec-9f74b39118e2 service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:47.110065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d695-2a55-414b-80ec-9f74b39118e2 service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:47.110340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d695-2a55-414b-80ec-9f74b39118e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:47.120519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125d695-2a55-414b-80ec-9f74b39118e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:47.122192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125d695-2a55-414b-80ec-9f74b39118e2 service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:47.130511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1178/2365] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:20:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:20:49.555914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:49.557788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:49.645105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a584b-d92a-495b-b4b0-14c1c07cfcd5 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:49.650032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:49.651020 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:49.651367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:49.651724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] PATH_INFO: `/v3/users/5e240eb390854a1bbe3125854f4d059e` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:49.652984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e240eb390854a1bbe3125854f4d059e)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:49.655087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:49.740620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e240eb390854a1bbe3125854f4d059e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:49.741633 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e240eb390854a1bbe3125854f4d059e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:49.750975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e240eb390854a1bbe3125854f4d059e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:49.751683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e240eb390854a1bbe3125854f4d059e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:49.772488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e240eb390854a1bbe3125854f4d059e'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:49.775350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:49.806040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1188/2366] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:20:49 2022] DELETE /identity/v3/users/5e240eb390854a1bbe3125854f4d059e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:49.811011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:49.811563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:49.811563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:49.836589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:49.838052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:49.930887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1179/2367] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:49.939791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:49.941506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:50.043473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14369d54-6ef0-46ed-9c09-302a02496516 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:50.047289 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05e4dd-9482-4ee8-b5ec-89293671afcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:50.047993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05e4dd-9482-4ee8-b5ec-89293671afcf service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:50.048262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05e4dd-9482-4ee8-b5ec-89293671afcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:50.048584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05e4dd-9482-4ee8-b5ec-89293671afcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:50.071279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05e4dd-9482-4ee8-b5ec-89293671afcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:50.072788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05e4dd-9482-4ee8-b5ec-89293671afcf service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:50.098937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1189/2368] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:50.588556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:50.592051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:50.628041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16482bf8-39f5-4bc1-8579-d1f75ee84a02 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:50.645148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:50.648649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:50.650219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:50.651302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] PATH_INFO: `/v3/projects/6d069f71a9d74f48b4aa79290bf6e3f1` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:50.655387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d069f71a9d74f48b4aa79290bf6e3f1)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:50.664297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:50.794356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d069f71a9d74f48b4aa79290bf6e3f1', 'request_id': 'req-b43bd249-7455-4821-a24e-ba46a6bf455e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:50.809529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d069f71a9d74f48b4aa79290bf6e3f1', 'request_id': 'req-b43bd249-7455-4821-a24e-ba46a6bf455e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:50.827229 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d069f71a9d74f48b4aa79290bf6e3f1', 'request_id': 'req-b43bd249-7455-4821-a24e-ba46a6bf455e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:50.836508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:50.853343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] The token cache is being invalidate because project 6d069f71a9d74f48b4aa79290bf6e3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:20:50.854021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:50.855398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-b43bd249-7455-4821-a24e-ba46a6bf455e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:20:50.858225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1180/2369] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:20:50 2022] DELETE /identity/v3/projects/6d069f71a9d74f48b4aa79290bf6e3f1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:50.863003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:50.863703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:50.863703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:50.895526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:50.898285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:51.012740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1190/2370] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:51.021266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.021544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.021818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.044090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:51.045319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:51.058017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.065564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1181/2371] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:51.069610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc25d4-61d3-45f8-9062-0a9691925469 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.076436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.076811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.077093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.077369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.078137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:20:51.078444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.080122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.114786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.128434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4fce0-0342-45b3-86d3-1b18e9dd6571 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.131462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1191/2372] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:20:51.141743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.142424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.142711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.143319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.144179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.146440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.191209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.197051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1182/2373] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:20:51 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:51.202216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5dd60-c396-4018-8706-ee4a080585c2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.331598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.332779 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.333125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.333511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.334352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.336740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.366214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1192/2374] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:20:51 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:20:51.367691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.380354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79006c39-410b-4216-a418-84a29e34862e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.389369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.390033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.390326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.390606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.391574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.393406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.404818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1183/2375] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:51.408709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.420922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449de31-81f2-4987-8d96-a520513bd56e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.427269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02a997-2bf9-4b5c-9f61-8cd3d71f52c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.428285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02a997-2bf9-4b5c-9f61-8cd3d71f52c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.428410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02a997-2bf9-4b5c-9f61-8cd3d71f52c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.428665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02a997-2bf9-4b5c-9f61-8cd3d71f52c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.429574 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02a997-2bf9-4b5c-9f61-8cd3d71f52c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.431488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02a997-2bf9-4b5c-9f61-8cd3d71f52c5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.440123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:20:51.448353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1193/2376] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:20:51.453256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.489884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d261d33-9d48-4639-9536-ba464230d2b0 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.502448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.503599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.504062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.504512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] PATH_INFO: `/v3/projects/f1756213e80b4aa08c94c5bcdfa4ba54/users/86ea0178875849a786b1e486b064eac3/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.528066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1756213e80b4aa08c94c5bcdfa4ba54, user_id=86ea0178875849a786b1e486b064eac3, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.531598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.558230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1184/2377] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:20:51 2022] PUT /identity/v3/projects/f1756213e80b4aa08c94c5bcdfa4ba54/users/86ea0178875849a786b1e486b064eac3/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:20:51.568797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af192fb-f579-4b02-be74-3dca5d533eac None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.569108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af192fb-f579-4b02-be74-3dca5d533eac None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.569407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af192fb-f579-4b02-be74-3dca5d533eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.602626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6af192fb-f579-4b02-be74-3dca5d533eac None None] MFA Rules not processed for user `86ea0178875849a786b1e486b064eac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:51.604248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6af192fb-f579-4b02-be74-3dca5d533eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:51.675047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1194/2378] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:20:51.687325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:20:51.689433 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:51.772248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb02ff-3367-4509-9cd0-e7d86a5c06a7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:20:51.776807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:20:51.778255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:51.778559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:51.778927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:51.788258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:20:51.789857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:20:51.808400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1185/2379] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:20:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:20:57.269401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8b4b-b933-4522-8e01-385e7a89c2c5 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:20:57.270141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8b4b-b933-4522-8e01-385e7a89c2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:20:57.270141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8b4b-b933-4522-8e01-385e7a89c2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:20:57.293530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-16bc8b4b-b933-4522-8e01-385e7a89c2c5 None None] MFA Rules not processed for user `36c5458a18f146be87207e0f6dd598eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:20:57.295316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-16bc8b4b-b933-4522-8e01-385e7a89c2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:20:57.375992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1195/2380] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:20:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:21:01.803464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=1,keystone:SELECT=64,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:02.554368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:02.643746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97c362-2ae2-4621-88cd-b58301e08293 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:02.648551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:02.649618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:02.650069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:02.650519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] PATH_INFO: `/v3/users/86ea0178875849a786b1e486b064eac3` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:02.651891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ea0178875849a786b1e486b064eac3)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:02.654489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:02.721170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ea0178875849a786b1e486b064eac3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:02.721820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ea0178875849a786b1e486b064eac3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:02.730147 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ea0178875849a786b1e486b064eac3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:02.731405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ea0178875849a786b1e486b064eac3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:02.741233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ea0178875849a786b1e486b064eac3'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:02.767445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1186/2381] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:21:02 2022] DELETE /identity/v3/users/86ea0178875849a786b1e486b064eac3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:21:02.771910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:02.772458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:02.772549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:02.795536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:02.797241 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:02.870244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1196/2382] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:02.882502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:02.885346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:03.062292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e935-e879-4c82-bd3b-580a430b5b3d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:03.075415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2d3d4-36c9-4517-a3ee-582b5128baf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:03.075415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2d3d4-36c9-4517-a3ee-582b5128baf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:03.075415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2d3d4-36c9-4517-a3ee-582b5128baf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:03.075415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2d3d4-36c9-4517-a3ee-582b5128baf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:03.085691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2d3d4-36c9-4517-a3ee-582b5128baf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:03.087422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2d3d4-36c9-4517-a3ee-582b5128baf0 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:03.114196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1187/2383] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:03.409281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:03.410890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:03.427150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710f829-f1a3-4eda-822b-f4638b3c12fd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:03.438662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:03.439984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:03.440629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:03.441025 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] PATH_INFO: `/v3/projects/f1756213e80b4aa08c94c5bcdfa4ba54` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:03.442414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1756213e80b4aa08c94c5bcdfa4ba54)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:03.444366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:03.486272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:03.503376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1756213e80b4aa08c94c5bcdfa4ba54', 'request_id': 'req-fd53aaba-3ac9-4afe-ade1-3755a6f79260'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:03.513740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1756213e80b4aa08c94c5bcdfa4ba54', 'request_id': 'req-fd53aaba-3ac9-4afe-ade1-3755a6f79260'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:03.520588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1756213e80b4aa08c94c5bcdfa4ba54', 'request_id': 'req-fd53aaba-3ac9-4afe-ade1-3755a6f79260'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:03.533364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] The token cache is being invalidate because project f1756213e80b4aa08c94c5bcdfa4ba54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:21:03.533656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:03.534372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:03.535873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1197/2384] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:21:03 2022] DELETE /identity/v3/projects/f1756213e80b4aa08c94c5bcdfa4ba54 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:21:10.418569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:10.419541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:10.419832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:10.439804 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:10.441155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:10.459435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:10.501882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1188/2385] 10.222.0.61 () {58 vars in 967 bytes} [Tue Jun 7 13:21:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:10.514516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:10.516159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:10.593074 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53aaba-3ac9-4afe-ade1-3755a6f79260 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:10.598547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:10.599221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:10.599504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:10.599850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:10.608670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:10.610243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:10.619125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1198/2386] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:21:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:21:12.530920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:12.532358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:12.603116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18107f69-bab4-4b24-8861-e4e63470df8b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:12.608494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:12.609382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:12.609672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:12.609951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] PATH_INFO: `/v3/users/51d5d87476eb4c8d8ceacf022fe4135d` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:12.611217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d5d87476eb4c8d8ceacf022fe4135d)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:12.613205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:12.670894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d5d87476eb4c8d8ceacf022fe4135d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:12.671595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d5d87476eb4c8d8ceacf022fe4135d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:12.678973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d5d87476eb4c8d8ceacf022fe4135d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:12.679643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d5d87476eb4c8d8ceacf022fe4135d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:12.687539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-33c3bb89-a3ba-4905-b884-2a87cf4d24af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d5d87476eb4c8d8ceacf022fe4135d'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:12.705397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1189/2387] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:21:12 2022] DELETE /identity/v3/users/51d5d87476eb4c8d8ceacf022fe4135d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:21:12.884984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:12.896200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031111-9956-4b80-a358-6bfbe362712e service swift] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:12.952295 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:12.953153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:12.953436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:12.953712 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] PATH_INFO: `/v3/projects/d1aac41f4f074858ba560987a158c38b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:12.954940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1aac41f4f074858ba560987a158c38b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:12.956822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.002076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1aac41f4f074858ba560987a158c38b', 'request_id': 'req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:13.003705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:13.010106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1aac41f4f074858ba560987a158c38b', 'request_id': 'req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:13.017193 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1aac41f4f074858ba560987a158c38b', 'request_id': 'req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:13.027868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] The token cache is being invalidate because project d1aac41f4f074858ba560987a158c38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:21:13.028153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:13.028799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-6c6d573e-cfa2-487a-a6a4-6a32ba4d4417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:13.030339 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1199/2388] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:21:12 2022] DELETE /identity/v3/projects/d1aac41f4f074858ba560987a158c38b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:21:13.035349 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.035635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.035908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.053041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:13.054235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:13.097682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:13.117369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1190/2389] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:21:13.125410 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.125701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.125983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.144643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:13.145846 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:13.154068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1200/2390] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:13.158243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.168398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5efaf-3954-4f69-a352-ca43048418e2 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.173057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.173715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.173991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.174259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.175013 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:21:13.175318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.176900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.192369 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1191/2391] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:21:13.193439 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.203364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd81d67-1096-404b-b848-dc21f869699b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.208063 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.208725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.208984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.209253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.209944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.211942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.240731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1201/2392] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:21:13 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:21:13.244780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.255567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3987f3-9528-4eb1-8113-4c25271afded admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.312375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.313032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.313319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.313597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.314297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.316220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.337242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1192/2393] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:21:13 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:21:13.341323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.351831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37ea84-7df4-4495-b2ef-04fb68c5dcfc admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.356372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.357536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.357819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.358091 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.358813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.360425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.370520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1202/2394] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:21:13.375045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.385081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a95f0f-f585-4445-a1a2-7c6d9f956807 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.389614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeed74f-8279-47a0-9e27-6fbad76bc2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.390242 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed74f-8279-47a0-9e27-6fbad76bc2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.390523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed74f-8279-47a0-9e27-6fbad76bc2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.390797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeed74f-8279-47a0-9e27-6fbad76bc2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.391551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed74f-8279-47a0-9e27-6fbad76bc2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.393125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeed74f-8279-47a0-9e27-6fbad76bc2ae admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.402976 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1193/2395] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:21:13.406804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.416748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09377b74-b6c4-46bd-bf0d-74fc033c89e4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.421990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.423021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.423377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.423726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] PATH_INFO: `/v3/projects/2dd6764f00464e11b05f9e56c55b971c/users/91ab47f13a5b49cfac0165656e285444/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.439335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd6764f00464e11b05f9e56c55b971c, user_id=91ab47f13a5b49cfac0165656e285444, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.441993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.454484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1203/2396] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:21:13 2022] PUT /identity/v3/projects/2dd6764f00464e11b05f9e56c55b971c/users/91ab47f13a5b49cfac0165656e285444/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:21:13.464633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.464897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.465176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.492022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] MFA Rules not processed for user `91ab47f13a5b49cfac0165656e285444`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:13.493215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:13.554795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1194/2397] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:21:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:13.566575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:13.568297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:13.624965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055b59-43e8-44d0-a06d-15a5876575da admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:13.628596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca2a05-30cf-4d7a-9227-74ed6031678f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:13.629232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2a05-30cf-4d7a-9227-74ed6031678f service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:13.629513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2a05-30cf-4d7a-9227-74ed6031678f service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:13.629788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2a05-30cf-4d7a-9227-74ed6031678f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:13.637998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca2a05-30cf-4d7a-9227-74ed6031678f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:13.639459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca2a05-30cf-4d7a-9227-74ed6031678f service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:13.648145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1204/2398] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:14.565104 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:14.566511 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:14.611876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:14.624403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc972bd5-f4cb-4c47-8838-f3251ff9b60c None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:14.627933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:14.628563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:14.628846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:14.629117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:14.630871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:14.692267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:14.693710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:14.703365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1195/2399] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:23.644145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=59 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:24.699014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=31 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:30.910167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:30.910627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:30.910807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:30.933303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:30.934482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:30.993706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1205/2400] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:21:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:31.016120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:31.017600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:31.088187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273eb1a-ddab-4de7-860c-e64865a9b70f service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:31.093279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3b803-1487-418f-85c6-793bc496b708 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:31.093931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b803-1487-418f-85c6-793bc496b708 service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:31.094220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b803-1487-418f-85c6-793bc496b708 service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:31.094499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b803-1487-418f-85c6-793bc496b708 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:31.102704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3b803-1487-418f-85c6-793bc496b708 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:31.104341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3b803-1487-418f-85c6-793bc496b708 service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:31.112447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1196/2401] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:21:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:21:40.987200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=25 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:41.109222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=32 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:43.161759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:43.163152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:43.254757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90864954-35dc-4fab-aaf8-ea2a4f74f512 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:43.259814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:43.260453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:43.260737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:43.261011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:43.262797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:43.274838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:43.276463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a2d20-1927-4508-bb51-f2c93c5f8aa7 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:43.287003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1206/2402] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:45.168271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bdf37-ab28-4a75-b9c1-0592c97cacc7 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.168846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bdf37-ab28-4a75-b9c1-0592c97cacc7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.168959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740bdf37-ab28-4a75-b9c1-0592c97cacc7 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.170399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1197/2403] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:21:45.176715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336bcc2f-8505-4a4d-9429-050f343167cf None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.177184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336bcc2f-8505-4a4d-9429-050f343167cf None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.177296 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336bcc2f-8505-4a4d-9429-050f343167cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.215043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-336bcc2f-8505-4a4d-9429-050f343167cf None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:45.216165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-336bcc2f-8505-4a4d-9429-050f343167cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:45.284291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1207/2404] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:45.296267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bdf37-ab28-4a75-b9c1-0592c97cacc7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:45.298482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-740bdf37-ab28-4a75-b9c1-0592c97cacc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:45.377680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740bdf37-ab28-4a75-b9c1-0592c97cacc7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:45.381249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2d196-fe95-4286-a5b7-fc7f82d2a38b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:45.381884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2d196-fe95-4286-a5b7-fc7f82d2a38b service glance] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.382170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2d196-fe95-4286-a5b7-fc7f82d2a38b service glance] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.382453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2d196-fe95-4286-a5b7-fc7f82d2a38b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.393034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2d196-fe95-4286-a5b7-fc7f82d2a38b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:45.394449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2d196-fe95-4286-a5b7-fc7f82d2a38b service glance] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:45.404950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1198/2405] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:45.470425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3aaceb-6b1c-4173-a1f0-45aa0ba5ed6b None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.470640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3aaceb-6b1c-4173-a1f0-45aa0ba5ed6b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.470958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3aaceb-6b1c-4173-a1f0-45aa0ba5ed6b None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.472382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1208/2406] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:21:45.477479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6317350-7db7-4b27-8700-c6e29482eadf None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.477763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6317350-7db7-4b27-8700-c6e29482eadf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.478037 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6317350-7db7-4b27-8700-c6e29482eadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.495988 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c6317350-7db7-4b27-8700-c6e29482eadf None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:45.497183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c6317350-7db7-4b27-8700-c6e29482eadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:45.504084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1199/2407] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:45.515316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3aaceb-6b1c-4173-a1f0-45aa0ba5ed6b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:45.526736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3aaceb-6b1c-4173-a1f0-45aa0ba5ed6b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:45.530290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35201c-d001-45bf-943b-a539a7a8d494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:45.530947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35201c-d001-45bf-943b-a539a7a8d494 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.531231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35201c-d001-45bf-943b-a539a7a8d494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.531501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35201c-d001-45bf-943b-a539a7a8d494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.538348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35201c-d001-45bf-943b-a539a7a8d494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:45.539784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35201c-d001-45bf-943b-a539a7a8d494 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:45.547792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1209/2408] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:45.631247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054c77-624f-435c-8130-ba428d428dc8 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.631538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054c77-624f-435c-8130-ba428d428dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.631812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86054c77-624f-435c-8130-ba428d428dc8 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.633225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1200/2409] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:21:45.638430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570344b-1212-4c2a-b5bb-166dc3705296 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.638722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570344b-1212-4c2a-b5bb-166dc3705296 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.639027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570344b-1212-4c2a-b5bb-166dc3705296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.658673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8570344b-1212-4c2a-b5bb-166dc3705296 None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:45.659908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8570344b-1212-4c2a-b5bb-166dc3705296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:45.678123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1210/2410] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:45.688620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86054c77-624f-435c-8130-ba428d428dc8 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:45.699537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86054c77-624f-435c-8130-ba428d428dc8 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:45.703572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:45.704190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:45.704466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:45.704732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:45.714456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:45.715908 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:45.727670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1201/2411] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:48.320700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:48.321173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:48.321267 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:48.339338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:48.340537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:48.347901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1211/2412] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:21:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:48.359734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:48.370759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e34d02-0ab4-4dde-b380-c8170e1617e7 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:48.376951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c7511-47fe-489e-a549-ef64191b26ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:48.377595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c7511-47fe-489e-a549-ef64191b26ba service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:48.377874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c7511-47fe-489e-a549-ef64191b26ba service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:48.378145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c7511-47fe-489e-a549-ef64191b26ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:48.386675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c7511-47fe-489e-a549-ef64191b26ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:48.388285 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c7511-47fe-489e-a549-ef64191b26ba service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:48.396799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1202/2413] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:21:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:21:53.942458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:54.023559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:54.081060 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d592356-861f-49f9-b0b4-05a4cdd81ea7 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:54.086278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:54.087983 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:54.088275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:54.088556 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] PATH_INFO: `/v3/users/91ab47f13a5b49cfac0165656e285444` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:54.089908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ab47f13a5b49cfac0165656e285444)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:54.091923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:54.147971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ab47f13a5b49cfac0165656e285444'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.148619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ab47f13a5b49cfac0165656e285444'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.156501 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ab47f13a5b49cfac0165656e285444'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.157805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ab47f13a5b49cfac0165656e285444'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.169458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ab47f13a5b49cfac0165656e285444'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.182497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1212/2414] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:21:53 2022] DELETE /identity/v3/users/91ab47f13a5b49cfac0165656e285444 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:21:54.187419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:54.188001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:54.188001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:54.207172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:21:54.208679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:54.278983 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1203/2415] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:21:54.288679 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:54.290527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:54.357082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb651fe2-89cf-4dac-9f26-c5f99eae8e30 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:54.360911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d289ea-afc8-4137-bc99-aa9bb928a861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:54.361605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d289ea-afc8-4137-bc99-aa9bb928a861 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:54.361898 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d289ea-afc8-4137-bc99-aa9bb928a861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:54.362216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d289ea-afc8-4137-bc99-aa9bb928a861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:54.379655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d289ea-afc8-4137-bc99-aa9bb928a861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:54.381126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d289ea-afc8-4137-bc99-aa9bb928a861 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:54.392551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1213/2416] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:21:54.810296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:21:54.812552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:21:54.833596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9362c-b024-4c0e-b9dd-e62649e86573 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:21:54.842688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:21:54.842688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:21:54.842688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:21:54.842688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] PATH_INFO: `/v3/projects/2dd6764f00464e11b05f9e56c55b971c` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:21:54.843667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd6764f00464e11b05f9e56c55b971c)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:21:54.846923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:21:54.910002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:21:54.914274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd6764f00464e11b05f9e56c55b971c', 'request_id': 'req-fba35984-dd5c-45fe-9e67-516e2d52ca1f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.926282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd6764f00464e11b05f9e56c55b971c', 'request_id': 'req-fba35984-dd5c-45fe-9e67-516e2d52ca1f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.937092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd6764f00464e11b05f9e56c55b971c', 'request_id': 'req-fba35984-dd5c-45fe-9e67-516e2d52ca1f'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.949613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] The token cache is being invalidate because project 2dd6764f00464e11b05f9e56c55b971c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:21:54.949900 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.950552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:21:54.952134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1204/2417] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:21:54 2022] DELETE /identity/v3/projects/2dd6764f00464e11b05f9e56c55b971c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:22:04.387571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:22:04.946364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:22:13.912372 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293698d3-d8d3-4582-8fcb-e71ddd5e03c1 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:22:13.912769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293698d3-d8d3-4582-8fcb-e71ddd5e03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:22:13.912938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293698d3-d8d3-4582-8fcb-e71ddd5e03c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:22:13.933143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-293698d3-d8d3-4582-8fcb-e71ddd5e03c1 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:22:13.934502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-293698d3-d8d3-4582-8fcb-e71ddd5e03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:22:14.013970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1214/2418] 10.222.0.61 () {58 vars in 967 bytes} [Tue Jun 7 13:22:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:22:14.026963 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:22:14.028346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:22:14.096394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba35984-dd5c-45fe-9e67-516e2d52ca1f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:22:14.101841 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43952ba5-d17d-41cd-b4e9-37d3edee3a8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:22:14.102479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43952ba5-d17d-41cd-b4e9-37d3edee3a8f service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:22:14.102765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43952ba5-d17d-41cd-b4e9-37d3edee3a8f service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:22:14.103333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43952ba5-d17d-41cd-b4e9-37d3edee3a8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:22:14.111253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43952ba5-d17d-41cd-b4e9-37d3edee3a8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:22:14.112824 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43952ba5-d17d-41cd-b4e9-37d3edee3a8f service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:22:14.120790 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1205/2419] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:22:17.636291 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:22:17.636812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:22:17.636919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:22:17.655103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] MFA Rules not processed for user `49c60c4956f8479fbfe93f9f85a8d497`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:22:17.656305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:22:17.713597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1215/2420] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:22:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:22:24.117646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=32 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:22:27.706371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=50 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:22:27.979208 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3af14-dab6-4c52-86d9-f31fbc45932b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:22:27.979725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3af14-dab6-4c52-86d9-f31fbc45932b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:22:27.979825 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3af14-dab6-4c52-86d9-f31fbc45932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:22:27.996939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9fa3af14-dab6-4c52-86d9-f31fbc45932b None None] MFA Rules not processed for user `dcf5286e9ca14f56905e6e247e411896`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:22:27.998121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9fa3af14-dab6-4c52-86d9-f31fbc45932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:22:28.050262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1206/2421] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:22:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:22:38.043812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=25 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:22:59.306094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:22:59.307713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:22:59.364857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d50985-ed6c-4f09-ad31-84cf836f2629 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:22:59.368544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:22:59.369195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:22:59.369510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:22:59.369789 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:22:59.371628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:22:59.443901 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:22:59.445343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:22:59.456196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1216/2422] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:22:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:02.494542 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:02.495097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:02.495202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:02.512721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] MFA Rules not processed for user `326e0d70244f46059664a135aefab0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:02.513912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:02.566576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1207/2423] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:23:02.601966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:02.603389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:02.659663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a313318-c9e2-45cc-b468-e8ae34336aea service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:02.663200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:02.663834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:02.664112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:02.664386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:02.666149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:02.679607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:02.681026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:02.691779 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1217/2424] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:02.952044 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:02.961424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20806db-ebf7-44c2-9a84-0ba165949598 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:02.964951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:02.965584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:02.965869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:02.966150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:02.967929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:02.977156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:02.978613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:02.986941 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1208/2425] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:03.389734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:03.391169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:03.438105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:03.461152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c4126-c738-47c2-84f2-6b0462a0aaec service glance] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:03.466238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:03.466912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:03.467197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:03.467471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:03.469216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:03.481257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:03.482869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:03.492612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1218/2426] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:12.248292 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:12.249673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:12.305406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66c5a0-ed90-4621-b266-d58a58218087 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:12.309047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:12.309685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:12.309970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:12.310249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:12.312058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:12.323191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:12.324643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:12.333608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1209/2427] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:13.488463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=12 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:15.766016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:15.767481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:15.783221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27c98f-3627-44ef-ae5a-63da13746529 service placement] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:15.788615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:15.789249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:15.789532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:15.789812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:15.791621 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:15.801944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:15.803575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:15.812413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1219/2428] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:19.568466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:19.570503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:19.586884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defb28f-3609-4d12-9727-96e6584739ca service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:19.590526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:19.591195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:19.591476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:19.591748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:19.593493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:19.604782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:19.606560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:19.615132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1210/2429] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:20.955335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:20.956995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:20.984185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0c84-a157-4016-aff5-4ad808b60eb4 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:20.990721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:20.991677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:20.992076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:20.992421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:20.994519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:21.010116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:21.012039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:21.025456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1220/2430] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:29.618881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=63 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:31.019282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=12 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:35.766792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:35.775435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290a344-68cb-4252-89db-5f321d2e5da1 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:35.778982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:35.779615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:35.779899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:35.780176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:35.781897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:35.845098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:35.846526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:35.856318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1211/2431] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:39.985533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:39.987612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:40.001759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692c757-5e2f-481c-9326-18ac9e737f83 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:40.007105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:40.007755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:40.008033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:40.009519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:40.011332 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:40.023514 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:40.025678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:40.044393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1221/2432] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:40.107356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:40.108875 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:40.121611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab6a27-3f6b-496f-9faf-c0320b622b26 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:40.126320 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:40.127225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:40.127508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:40.127779 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] PATH_INFO: `/v3/users/dcf5286e9ca14f56905e6e247e411896` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:40.128952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf5286e9ca14f56905e6e247e411896)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:40.130884 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:40.181673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf5286e9ca14f56905e6e247e411896'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.182335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf5286e9ca14f56905e6e247e411896'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.190167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf5286e9ca14f56905e6e247e411896'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.190855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf5286e9ca14f56905e6e247e411896'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.199000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-bc71d7fe-cbb9-4df9-b0c6-6c92dcf9bb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf5286e9ca14f56905e6e247e411896'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.216767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1212/2433] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:23:40 2022] DELETE /identity/v3/users/dcf5286e9ca14f56905e6e247e411896 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:23:40.564414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:40.575924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789bf24-b383-4fd3-927c-412e4dd02ad2 service placement] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:40.642021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:40.642938 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:40.643208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:40.643483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] PATH_INFO: `/v3/projects/7c615cd1e80a42e0ac2aff126cbb1455` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:40.644666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c615cd1e80a42e0ac2aff126cbb1455)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:40.646568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:40.701769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c615cd1e80a42e0ac2aff126cbb1455', 'request_id': 'req-3afc920e-78de-4900-b9ba-2ce52e5f9761'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.715184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c615cd1e80a42e0ac2aff126cbb1455', 'request_id': 'req-3afc920e-78de-4900-b9ba-2ce52e5f9761'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.724656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c615cd1e80a42e0ac2aff126cbb1455', 'request_id': 'req-3afc920e-78de-4900-b9ba-2ce52e5f9761'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.739899 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] The token cache is being invalidate because project 7c615cd1e80a42e0ac2aff126cbb1455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:23:40.740214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.741402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3afc920e-78de-4900-b9ba-2ce52e5f9761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:23:40.746875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1222/2434] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:23:40 2022] DELETE /identity/v3/projects/7c615cd1e80a42e0ac2aff126cbb1455 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:23:40.748534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:40.748831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:40.749116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:40.778250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:40.779887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:40.885678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1213/2435] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:23:40.893354 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:40.893692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:40.893988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:40.936640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:40.937911 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:40.964737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1223/2436] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:23:40.969497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:40.991082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688b0a0-ce04-464f-8556-1663a84ba8c1 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:40.998479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:40.999476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:40.999969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.001073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.002051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:23:41.002587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.004466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.008160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:41.048280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1214/2437] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:23:41.051848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:41.062795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239c09b-081b-4662-b571-33ef08dd2f41 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:41.069865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:41.070865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.071319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.071767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.072625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.074706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.110947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1224/2438] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:23:41 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:23:41.115022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:41.129125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eb08e-033b-4fd0-b809-616646681af1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:41.195183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:41.195893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.196178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.196454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.197139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.199072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.223341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1215/2439] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:23:41 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:23:41.227980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:41.239361 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314a827-afe4-47f7-aa18-1675c6017363 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:41.245048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:41.245959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.246322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.246754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.247588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.249273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.261399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1225/2440] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:23:41.265421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:41.277124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c3408-2fd9-4016-bf19-95247f10be46 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:41.283628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5293e-b05d-49b7-9500-9698155154b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:41.284303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5293e-b05d-49b7-9500-9698155154b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.284588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5293e-b05d-49b7-9500-9698155154b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.284866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5293e-b05d-49b7-9500-9698155154b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.285594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5293e-b05d-49b7-9500-9698155154b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.287368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5293e-b05d-49b7-9500-9698155154b1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.302724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1216/2441] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:23:41.303916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:41.314456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb38796-71b4-46f3-b119-a9c3feda8762 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:41.323099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:41.324035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.324318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.324591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] PATH_INFO: `/v3/projects/5541835571f64b4dbec6cabbbd6b13ca/users/a5766b86063f4955a642ae59e28ee226/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.360922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5541835571f64b4dbec6cabbbd6b13ca, user_id=a5766b86063f4955a642ae59e28ee226, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.364636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.393699 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1226/2442] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:23:41 2022] PUT /identity/v3/projects/5541835571f64b4dbec6cabbbd6b13ca/users/a5766b86063f4955a642ae59e28ee226/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:23:41.398666 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265634-6cf8-4548-9b0d-b06494310c0a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.399863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265634-6cf8-4548-9b0d-b06494310c0a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.400144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265634-6cf8-4548-9b0d-b06494310c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.454260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-5e265634-6cf8-4548-9b0d-b06494310c0a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:41.459548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5e265634-6cf8-4548-9b0d-b06494310c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:41.619221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1217/2443] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:23:41.633282 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:41.636419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:41.662551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:41.724585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c38fd-f03e-449d-a6d9-8fb0632b6588 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:41.729375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:41.730090 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:41.730376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:41.730676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:41.740915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:41.742445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:41.753466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1227/2444] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:42.727619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f34afb-781e-49fc-a6ab-65902b3dcfb8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:42.728184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f34afb-781e-49fc-a6ab-65902b3dcfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:42.728184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f34afb-781e-49fc-a6ab-65902b3dcfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:42.758905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-94f34afb-781e-49fc-a6ab-65902b3dcfb8 None None] MFA Rules not processed for user `36c5458a18f146be87207e0f6dd598eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:42.761870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-94f34afb-781e-49fc-a6ab-65902b3dcfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:42.856676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1218/2445] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:23:43.397709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:43.400258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:43.476711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef110611-cd42-4801-aafc-8453347f5a7c service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:43.482686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:43.483610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:43.489925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:43.489925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:43.489925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:43.505959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:43.511126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965cee1-55fb-4580-bc13-5d2bdf247219 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:43.539343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1228/2446] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:45.416633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:45.417174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:45.417423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:45.442014 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] MFA Rules not processed for user `49c60c4956f8479fbfe93f9f85a8d497`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:45.443615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:45.459757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:23:45.530267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1219/2447] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:23:47.445424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e5dd5-b806-425e-bf97-1ec5cabb42fb None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:47.446612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e5dd5-b806-425e-bf97-1ec5cabb42fb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:47.446612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e5dd5-b806-425e-bf97-1ec5cabb42fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:47.478893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-357e5dd5-b806-425e-bf97-1ec5cabb42fb None None] MFA Rules not processed for user `a5766b86063f4955a642ae59e28ee226`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:47.480278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-357e5dd5-b806-425e-bf97-1ec5cabb42fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:47.572375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1229/2448] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:23:47.586561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:47.588536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:47.713411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634be369-2a12-46fc-8f98-cda9526acdd4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:47.721757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:47.723176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:47.724051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:47.724409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:47.743922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:47.755985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:47.772227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1220/2449] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:49.852913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:49.854418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:49.854962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:49.879253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] MFA Rules not processed for user `a5766b86063f4955a642ae59e28ee226`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:23:49.881054 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:49.890379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1230/2450] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:23:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:23:55.280173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:23:55.281781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:55.297801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f4eb-da9f-4626-9d25-ee72a6fb1553 service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:23:55.303590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:23:55.304235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:23:55.304722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:23:55.305216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:23:55.307350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:23:55.320599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:23:55.322289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:23:55.334568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1221/2451] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:23:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:23:59.874543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=61 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:01.094555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:24:01.108292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a7f32-67d7-4e74-8d32-9f980c04184e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:24:01.113780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:24:01.114425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:01.114709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:01.115013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:01.116821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:24:01.129681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:24:01.131314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f524893-3cfa-463f-af57-755a06d3226c service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:24:01.142566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1231/2452] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:24:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:24:05.329783 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=57 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:11.138036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=6 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:18.294590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:24:18.307375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cefc44-c456-47dd-b1f1-8f78feaa0d60 service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:24:18.312888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:24:18.313521 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:18.313808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:18.314086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:18.315891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:24:18.373158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:24:18.374726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:24:18.383758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1222/2453] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:24:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:24:24.467832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e21467-42d7-4509-918f-74dee38545f9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:24.468335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e21467-42d7-4509-918f-74dee38545f9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:24.468433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e21467-42d7-4509-918f-74dee38545f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:24.486727 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a8e21467-42d7-4509-918f-74dee38545f9 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:24:24.487943 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a8e21467-42d7-4509-918f-74dee38545f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:24:24.543144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1232/2454] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:24:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:24:24.588980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:24:24.590438 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:24:24.659807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08fa6a-96dd-4f6d-b4e1-3e6073eb24b4 service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:24:24.666055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:24:24.666698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:24.667012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:24.667289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:24.675425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:24:24.677010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:24:24.684899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1223/2455] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:24:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:24:34.535993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=25 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:34.681748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=60 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:39.897224 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7e376-ac7e-4d44-999c-d7484712820a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:39.897691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7e376-ac7e-4d44-999c-d7484712820a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:39.897813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7e376-ac7e-4d44-999c-d7484712820a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:39.915996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8af7e376-ac7e-4d44-999c-d7484712820a None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:24:39.917174 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8af7e376-ac7e-4d44-999c-d7484712820a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:24:39.924221 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1233/2456] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:24:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:24:39.935877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:24:39.947225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadee82-14de-4bb5-994c-1b470cbba62b service swift] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:24:39.952221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:24:39.952863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:39.953129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:39.953486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:39.961587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:24:39.963745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:24:39.971698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1224/2457] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:24:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:24:49.910634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:49.968417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=6 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:24:55.576900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:55.577359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:55.577475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:55.594792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:24:55.595990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:24:55.602955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1234/2458] 10.222.0.61 () {58 vars in 967 bytes} [Tue Jun 7 13:24:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:24:55.615504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:24:55.626576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9fbea-1c53-42e3-9fbb-3d08ef4cbd34 service swift] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:24:55.631608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:24:55.632235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:24:55.632520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:24:55.632803 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:24:55.640625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:24:55.642187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:24:55.650074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1225/2459] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 13:25:05.589653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:05.646821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=6 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:13.870434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:13.879787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57590754-5fe8-487b-814c-b468d6c4d182 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:13.883365 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:13.884005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:13.884290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:13.884566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:13.887029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:13.898543 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:13.900008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:13.909924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1235/2460] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:17.576807 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:17.578374 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:17.592699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e81ba-9867-448b-ba7c-974dc1e49ac3 service swift] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:17.597205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:17.598066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:17.598347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:17.598622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] PATH_INFO: `/v3/users/49c60c4956f8479fbfe93f9f85a8d497` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:17.599829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c60c4956f8479fbfe93f9f85a8d497)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:17.601733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:17.649856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c60c4956f8479fbfe93f9f85a8d497'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.650527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c60c4956f8479fbfe93f9f85a8d497'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.657364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c60c4956f8479fbfe93f9f85a8d497'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.657989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c60c4956f8479fbfe93f9f85a8d497'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.665111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-d915f78f-319d-4c44-8cae-9e1fe5aa5774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c60c4956f8479fbfe93f9f85a8d497'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.683008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1226/2461] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:17 2022] DELETE /identity/v3/users/49c60c4956f8479fbfe93f9f85a8d497 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:17.850218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:17.860671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0439ee-d0f2-49f2-97f2-42bd02582656 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:17.918085 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:17.918972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:17.919263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:17.919539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] PATH_INFO: `/v3/projects/ec73e3d0e6f8476d81463ea6e11b62cd` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:17.920711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec73e3d0e6f8476d81463ea6e11b62cd)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:17.922648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:17.967960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec73e3d0e6f8476d81463ea6e11b62cd', 'request_id': 'req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.976152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec73e3d0e6f8476d81463ea6e11b62cd', 'request_id': 'req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.982565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec73e3d0e6f8476d81463ea6e11b62cd', 'request_id': 'req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.994178 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] The token cache is being invalidate because project ec73e3d0e6f8476d81463ea6e11b62cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:17.994483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.995708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-49c84fe2-4dfa-4ea7-a2c3-28ccac341779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:17.997462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1236/2462] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:17 2022] DELETE /identity/v3/projects/ec73e3d0e6f8476d81463ea6e11b62cd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:18.002090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.002383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.002660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.020519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:18.021756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:18.082308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1227/2463] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:18.090290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.090643 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.091022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.109750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:18.110974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:18.118792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1237/2464] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:18.123549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.133319 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f514c88-0436-461d-9207-4d56fa8b7717 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.137878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.138522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.138812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.139117 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.139842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:18.140150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.141812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.153416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1228/2465] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:25:18.157636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.167227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99313cd0-b95a-495b-9df1-0fc1bf5ac648 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.171842 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.172479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.172758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.173032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.173710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.175626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.210149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1238/2466] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:18.214142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.224214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b453-2175-433e-bee6-9493a97ae242 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.279879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.280534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.280816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.281092 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.281763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.283698 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.290161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:18.305432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1229/2467] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:25:18.309902 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.320120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f1724-7347-41f5-8e23-ead12bd3321c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.324733 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.325383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.325673 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.325950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.326678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.328307 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.338094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1239/2468] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:18.342313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.352154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b6294-719b-4b46-bf64-4f65ceda23d6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.357387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c72ac-527b-47cc-ac34-2224820db304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.358032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c72ac-527b-47cc-ac34-2224820db304 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.358315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c72ac-527b-47cc-ac34-2224820db304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.358592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c72ac-527b-47cc-ac34-2224820db304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.359610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c72ac-527b-47cc-ac34-2224820db304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.361193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c72ac-527b-47cc-ac34-2224820db304 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.370986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1230/2469] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:18.374661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.384406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243764-17d4-4722-8bcf-4fb09e19f9ed admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.389022 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.389881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.390160 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.390431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] PATH_INFO: `/v3/projects/ad3f2c7d9d8a457fb39f37d5250de0fd/users/80d6bc4644c2402486e3e6c477a8f1ab/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.405103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3f2c7d9d8a457fb39f37d5250de0fd, user_id=80d6bc4644c2402486e3e6c477a8f1ab, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.407780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.421155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1240/2470] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:18 2022] PUT /identity/v3/projects/ad3f2c7d9d8a457fb39f37d5250de0fd/users/80d6bc4644c2402486e3e6c477a8f1ab/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:18.432289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.432591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.432874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.458766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] MFA Rules not processed for user `80d6bc4644c2402486e3e6c477a8f1ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:18.459999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:18.523523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1231/2471] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:18.528348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.534031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:18.592441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9a877-aabe-4a94-8950-d1d2b3a1ca5c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.597195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.597882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.598171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.598453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.599180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.601087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.626370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1241/2472] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:18.631047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.641516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f6446-7d3b-4ef0-8b83-10d65786b5bf None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.698716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.699417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.699706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.699982 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.700678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.702596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.722591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1232/2473] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:18.726521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.737144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab880-cac7-4e53-ada3-8f8576461990 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.741769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.742415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.742695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.743008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.743755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.745335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.755610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1242/2474] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:18.759535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.769748 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b424f75-41e8-4498-b1d9-8184059bec13 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.774408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.775271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.775555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.776440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] PATH_INFO: `/v3/projects/a65b187d896343b2bb07343aec551c09/users/f71e4354125a4f22a56bf2117ce32576/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.790861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65b187d896343b2bb07343aec551c09, user_id=f71e4354125a4f22a56bf2117ce32576, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.793520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.806342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1233/2475] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:18 2022] PUT /identity/v3/projects/a65b187d896343b2bb07343aec551c09/users/f71e4354125a4f22a56bf2117ce32576/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:18.810243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.819974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fda46-ba18-4bad-9d77-29a4fddb9c87 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.874758 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410039bd-cc41-4c5a-8d7a-f358b4413955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.875426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410039bd-cc41-4c5a-8d7a-f358b4413955 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.875705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410039bd-cc41-4c5a-8d7a-f358b4413955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.875977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410039bd-cc41-4c5a-8d7a-f358b4413955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.876696 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410039bd-cc41-4c5a-8d7a-f358b4413955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.878274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410039bd-cc41-4c5a-8d7a-f358b4413955 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.888175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1243/2476] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:25:18.892344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:18.903042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dad34-9786-4177-84c9-3dacab01fbac admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:18.907701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:18.908562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.908847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.909123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] PATH_INFO: `/v3/projects/a65b187d896343b2bb07343aec551c09/users/f71e4354125a4f22a56bf2117ce32576/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.911094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65b187d896343b2bb07343aec551c09, user_id=f71e4354125a4f22a56bf2117ce32576, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:18.913721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:18.923750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1234/2477] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:18 2022] PUT /identity/v3/projects/a65b187d896343b2bb07343aec551c09/users/f71e4354125a4f22a56bf2117ce32576/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:18.933816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:18.934146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:18.934426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:18.960827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] MFA Rules not processed for user `f71e4354125a4f22a56bf2117ce32576`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:18.962011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:19.025572 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:19.030417 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1244/2478] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:19.041213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:19.042958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:19.065168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:19.103642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927e1d8-e131-4ce9-9133-0dd35ff9bbf3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:19.107441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292e982-dbd8-4c58-820a-899200f4da76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:19.108101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e982-dbd8-4c58-820a-899200f4da76 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:19.108383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e982-dbd8-4c58-820a-899200f4da76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:19.108675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292e982-dbd8-4c58-820a-899200f4da76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:19.119519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292e982-dbd8-4c58-820a-899200f4da76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:19.121430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292e982-dbd8-4c58-820a-899200f4da76 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:19.131330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1235/2479] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:19.158966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:19.219469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1972765-ae69-4a6a-b5a8-3ee5bfaf9897 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:19.224196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:19.225092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:19.225371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:19.225646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] PATH_INFO: `/v3/users/80d6bc4644c2402486e3e6c477a8f1ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:19.226853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d6bc4644c2402486e3e6c477a8f1ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:19.228800 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:19.276013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d6bc4644c2402486e3e6c477a8f1ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.276576 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d6bc4644c2402486e3e6c477a8f1ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.283223 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d6bc4644c2402486e3e6c477a8f1ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.283856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d6bc4644c2402486e3e6c477a8f1ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.295147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d6bc4644c2402486e3e6c477a8f1ab'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.308277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1245/2480] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:19 2022] DELETE /identity/v3/users/80d6bc4644c2402486e3e6c477a8f1ab => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:19.312709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:19.313003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:19.313277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:19.331202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:19.332385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:19.389022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1236/2481] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:19.398451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:19.399874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:19.457157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb835-f724-4719-a959-b32bfd4f216e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:19.460843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:19.461485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:19.461769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:19.462044 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:19.470967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:19.472387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:19.482166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1246/2482] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:19.758637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:19.760197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:19.772470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fd0b1-61bb-4a47-b5f1-d3c5cc16f0f0 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:19.777384 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:19.778321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:19.778686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:19.779026 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] PATH_INFO: `/v3/projects/ad3f2c7d9d8a457fb39f37d5250de0fd` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:19.780221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad3f2c7d9d8a457fb39f37d5250de0fd)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:19.782169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:19.828932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad3f2c7d9d8a457fb39f37d5250de0fd', 'request_id': 'req-0412ffaf-5404-4999-923d-b52ae7c53743'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.834912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad3f2c7d9d8a457fb39f37d5250de0fd', 'request_id': 'req-0412ffaf-5404-4999-923d-b52ae7c53743'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.842522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad3f2c7d9d8a457fb39f37d5250de0fd', 'request_id': 'req-0412ffaf-5404-4999-923d-b52ae7c53743'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.852811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] The token cache is being invalidate because project ad3f2c7d9d8a457fb39f37d5250de0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:19.853099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.853744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.855947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1237/2483] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:19 2022] DELETE /identity/v3/projects/ad3f2c7d9d8a457fb39f37d5250de0fd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:19.860128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:19.870031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3aa480-95b6-427c-9e69-b0bd8b1aa5fb service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:19.887968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:19.926882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:19.927750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:19.928034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:19.928311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] PATH_INFO: `/v3/users/f71e4354125a4f22a56bf2117ce32576` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:19.929492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71e4354125a4f22a56bf2117ce32576)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:19.931447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:19.977172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71e4354125a4f22a56bf2117ce32576'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.977809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71e4354125a4f22a56bf2117ce32576'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.985541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71e4354125a4f22a56bf2117ce32576'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.986156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71e4354125a4f22a56bf2117ce32576'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:19.997585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c455e260-ac13-4d0e-bcea-83ea62890dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71e4354125a4f22a56bf2117ce32576'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:20.009723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1247/2484] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:19 2022] DELETE /identity/v3/users/f71e4354125a4f22a56bf2117ce32576 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:20.269959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.271522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:20.285602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412ffaf-5404-4999-923d-b52ae7c53743 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.329432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:20.343307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.344167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.344451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.344722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] PATH_INFO: `/v3/projects/a65b187d896343b2bb07343aec551c09` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.345901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65b187d896343b2bb07343aec551c09)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.347869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.393769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65b187d896343b2bb07343aec551c09', 'request_id': 'req-9ca4cf2f-c96c-47ad-8993-d904e85cb340'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:20.399622 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65b187d896343b2bb07343aec551c09', 'request_id': 'req-9ca4cf2f-c96c-47ad-8993-d904e85cb340'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:20.407210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65b187d896343b2bb07343aec551c09', 'request_id': 'req-9ca4cf2f-c96c-47ad-8993-d904e85cb340'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:20.417656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] The token cache is being invalidate because project a65b187d896343b2bb07343aec551c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:20.417938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:20.418639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9ca4cf2f-c96c-47ad-8993-d904e85cb340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:20.420225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1238/2485] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:20 2022] DELETE /identity/v3/projects/a65b187d896343b2bb07343aec551c09 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:20.424984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.425274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.425554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.449071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:20.450277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:20.507756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1248/2486] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:20.515770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.516060 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.516340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.535449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:20.536625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:20.544649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1239/2487] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:20.548796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.559710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa611f48-18bc-48db-a618-62620a3e81cc None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.564488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.565148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.565442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.565709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.566473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:20.566753 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.568452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.581242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1249/2488] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:25:20.585533 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.594747 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00755c8b-8b29-4003-9bf3-7031147248b4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.599406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.600053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.600334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.600606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.601281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.603196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.629341 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1240/2489] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:20 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:20.633544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.644249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739ab99-b81e-47e6-96e4-58bf8146abbc admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.670483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:20.702962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.703639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.703925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.704200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.704883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.706808 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.725837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1250/2490] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:20 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:20.729986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.740478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e891fca-cdcf-4f38-8fb3-23297746c6ee admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.745562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.746214 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.746499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.746776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.747532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.749120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.759355 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1241/2491] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:20.763871 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.773765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9acd5-8a70-4f7b-b1cf-52f7f268f4d1 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.778386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be93f8-30ce-446d-9c8f-de242433d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.779069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be93f8-30ce-446d-9c8f-de242433d4ad admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.779355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be93f8-30ce-446d-9c8f-de242433d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.779632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be93f8-30ce-446d-9c8f-de242433d4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.780351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be93f8-30ce-446d-9c8f-de242433d4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.781927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be93f8-30ce-446d-9c8f-de242433d4ad admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.791534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1251/2492] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:25:20.795336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:20.805081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc585766-130f-4aa1-8522-26062f2c481c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:20.810330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:20.811235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.811518 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.811788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] PATH_INFO: `/v3/projects/a9ced85f52f543449a80826dd791c91d/users/5d92ce081a854bf3bbb8ab8853b2d970/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.827175 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ced85f52f543449a80826dd791c91d, user_id=5d92ce081a854bf3bbb8ab8853b2d970, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:20.829827 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:20.842503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1242/2493] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:20 2022] PUT /identity/v3/projects/a9ced85f52f543449a80826dd791c91d/users/5d92ce081a854bf3bbb8ab8853b2d970/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:20.853813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:20.854103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:20.854409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:20.881310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] MFA Rules not processed for user `5d92ce081a854bf3bbb8ab8853b2d970`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:20.882517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:20.945000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1252/2494] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:20.950125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.010574 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c568c-f1d8-46e7-b6de-66a5e3f7f1f1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.015095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.015738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.016021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.016294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.016966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.018873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.037263 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:21.047148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1243/2495] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:20 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:21.050766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.061969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fba943-e56a-4a04-987f-f776e8189c2c None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.116094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.116752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.117036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.117313 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.117994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.120569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.139376 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1253/2496] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:21 2022] POST /identity/v3/users => generated 307 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:21.143260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.153427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c7607-c6bd-4ea2-8b89-7e592d1e4d0c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.157980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.158625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.158947 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.159221 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.159933 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.161515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.172414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1244/2497] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:21.176230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.186620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a2a7b-3064-4b5d-9c10-0f30940ddcfc admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.191331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.192192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.192476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.192755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] PATH_INFO: `/v3/projects/23368469f17d4f58bff3ab483ba7cced/users/fa3e4b0902ee4082b7208bcae181cde5/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.207351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23368469f17d4f58bff3ab483ba7cced, user_id=fa3e4b0902ee4082b7208bcae181cde5, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.209988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.223134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1254/2498] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:21 2022] PUT /identity/v3/projects/23368469f17d4f58bff3ab483ba7cced/users/fa3e4b0902ee4082b7208bcae181cde5/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:21.227029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.238161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd77d4-ec06-49f4-8339-e0cf452baa79 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.292460 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f9482-0608-4b1d-9f04-582bf32eace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.293106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f9482-0608-4b1d-9f04-582bf32eace6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.293389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f9482-0608-4b1d-9f04-582bf32eace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.293664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f9482-0608-4b1d-9f04-582bf32eace6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.294373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f9482-0608-4b1d-9f04-582bf32eace6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.296061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f9482-0608-4b1d-9f04-582bf32eace6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.306650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1245/2499] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:21.311812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.318206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:21.323986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82e569-0207-4af8-9288-9639e9fe4fe7 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.330711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.331635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.331915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.332189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] PATH_INFO: `/v3/projects/23368469f17d4f58bff3ab483ba7cced/users/fa3e4b0902ee4082b7208bcae181cde5/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.334163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23368469f17d4f58bff3ab483ba7cced, user_id=fa3e4b0902ee4082b7208bcae181cde5, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.336897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.347892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1255/2500] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:21 2022] PUT /identity/v3/projects/23368469f17d4f58bff3ab483ba7cced/users/fa3e4b0902ee4082b7208bcae181cde5/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:21.358522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.358809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.359116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.384929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] MFA Rules not processed for user `fa3e4b0902ee4082b7208bcae181cde5`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:21.386154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:21.452934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1246/2501] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:21.464071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.465774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:21.522172 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b68d48-21af-4d05-9fdd-b3bed05935c4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.525939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.527114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.527396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.527664 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.536721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.538126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.548142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1256/2502] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:21.617210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:21.626932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4ef7e-5c92-41f6-b2b3-1033827af26b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:21.630520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002df9e-5ffb-43c0-a67f-91565eedf7be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:21.631185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002df9e-5ffb-43c0-a67f-91565eedf7be service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:21.631467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002df9e-5ffb-43c0-a67f-91565eedf7be service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:21.631737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002df9e-5ffb-43c0-a67f-91565eedf7be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:21.688365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002df9e-5ffb-43c0-a67f-91565eedf7be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:21.689822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002df9e-5ffb-43c0-a67f-91565eedf7be service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:21.699075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1247/2503] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:25.709401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:25.771229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d0974-20af-40cd-bcb4-e0ea6a99e0ba service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:25.775791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:25.776653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:25.776937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:25.777213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] PATH_INFO: `/v3/users/5d92ce081a854bf3bbb8ab8853b2d970` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:25.778425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d92ce081a854bf3bbb8ab8853b2d970)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:25.780388 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:25.826448 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d92ce081a854bf3bbb8ab8853b2d970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:25.827147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d92ce081a854bf3bbb8ab8853b2d970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:25.833915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d92ce081a854bf3bbb8ab8853b2d970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:25.834531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d92ce081a854bf3bbb8ab8853b2d970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:25.846701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d92ce081a854bf3bbb8ab8853b2d970'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:25.860517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1257/2504] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:25 2022] DELETE /identity/v3/users/5d92ce081a854bf3bbb8ab8853b2d970 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:25.864951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:25.865480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:25.865592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:25.877078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:25.883459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:25.884658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:25.941670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1248/2505] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:25.951103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:25.952521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:25.966402 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:26.010982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f0069-a302-45f4-b8fa-252a2200ad7b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:26.014636 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:26.015304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:26.015595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:26.015888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:26.024975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:26.027203 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:26.040233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1258/2506] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:26.309452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:26.311016 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:26.323826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c2321-82be-4539-a0a5-f5d4b3b63021 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:26.328589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:26.329457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:26.329744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:26.330020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] PATH_INFO: `/v3/projects/a9ced85f52f543449a80826dd791c91d` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:26.331266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9ced85f52f543449a80826dd791c91d)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:26.333186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:26.381670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9ced85f52f543449a80826dd791c91d', 'request_id': 'req-971ddc54-93f3-42e1-9f4d-0335482c7de5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.388386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9ced85f52f543449a80826dd791c91d', 'request_id': 'req-971ddc54-93f3-42e1-9f4d-0335482c7de5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.395642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9ced85f52f543449a80826dd791c91d', 'request_id': 'req-971ddc54-93f3-42e1-9f4d-0335482c7de5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.406692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] The token cache is being invalidate because project a9ced85f52f543449a80826dd791c91d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:26.407011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.407694 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.409217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1249/2507] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:26 2022] DELETE /identity/v3/projects/a9ced85f52f543449a80826dd791c91d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:26.413143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:26.424993 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945477e5-6d0a-4f27-b1e1-46759ce18563 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:26.480537 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:26.481322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:26.481603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:26.481874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] PATH_INFO: `/v3/users/fa3e4b0902ee4082b7208bcae181cde5` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:26.483092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3e4b0902ee4082b7208bcae181cde5)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:26.485353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:26.529485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3e4b0902ee4082b7208bcae181cde5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.530119 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3e4b0902ee4082b7208bcae181cde5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.537016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3e4b0902ee4082b7208bcae181cde5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.537745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3e4b0902ee4082b7208bcae181cde5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.549517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a6ec8316-f1b0-4a49-86d9-20486be1ec2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3e4b0902ee4082b7208bcae181cde5'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.562347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1259/2508] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:26 2022] DELETE /identity/v3/users/fa3e4b0902ee4082b7208bcae181cde5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:26.826308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:26.827754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:26.843998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ddc54-93f3-42e1-9f4d-0335482c7de5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:26.899638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:26.900487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:26.900767 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:26.901530 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] PATH_INFO: `/v3/projects/23368469f17d4f58bff3ab483ba7cced` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:26.902684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23368469f17d4f58bff3ab483ba7cced)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:26.904609 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:26.938045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:26.951279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23368469f17d4f58bff3ab483ba7cced', 'request_id': 'req-dbb9ff54-58bf-45f1-8bb6-b259b685a210'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.957298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23368469f17d4f58bff3ab483ba7cced', 'request_id': 'req-dbb9ff54-58bf-45f1-8bb6-b259b685a210'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.964849 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23368469f17d4f58bff3ab483ba7cced', 'request_id': 'req-dbb9ff54-58bf-45f1-8bb6-b259b685a210'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.975153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] The token cache is being invalidate because project 23368469f17d4f58bff3ab483ba7cced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:26.975441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.976059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-dbb9ff54-58bf-45f1-8bb6-b259b685a210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:26.977606 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1250/2509] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:26 2022] DELETE /identity/v3/projects/23368469f17d4f58bff3ab483ba7cced => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:26.982607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:26.983120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:26.983215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.000508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:27.001704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:27.031071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:27.061759 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1260/2510] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:27.070584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.070903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.071187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.090099 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:27.091315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:27.099130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1251/2511] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:27.103189 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.113019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15045595-80db-4cb6-8e4b-f441eb7295c2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.117631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.118281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.118561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.118855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.119593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:27.119927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.121522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.132764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1261/2512] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:25:27.137383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.146502 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62259fe5-1f29-43e6-bcdb-6402d143193d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.151797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.152436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.152719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.152989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.153673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.155602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.180638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1252/2513] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:25:27.184804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.194646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf75d7-9772-48a1-a1a3-0b8043abd4af admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.250754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.251436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.251721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.251995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.252682 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.254590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.274324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1262/2514] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:27.279072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.289340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4409114-d84b-4a4d-96e1-610d9e041292 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.293838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.294480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.294761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.295067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.295809 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.297488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.308611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1253/2515] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:27.312257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.322005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40efb-11fa-4e6b-a3f8-2740d74fc192 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.327228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b73a9e-5fef-4064-8d38-bd108ad3aa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.327868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b73a9e-5fef-4064-8d38-bd108ad3aa74 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.328153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b73a9e-5fef-4064-8d38-bd108ad3aa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.328426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b73a9e-5fef-4064-8d38-bd108ad3aa74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.329140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b73a9e-5fef-4064-8d38-bd108ad3aa74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.330703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b73a9e-5fef-4064-8d38-bd108ad3aa74 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.340397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1263/2516] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:27.344176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.354163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4010ae6-b1d6-482f-9d6e-c643f6ccad41 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.358731 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.359902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.360183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.360474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] PATH_INFO: `/v3/projects/b589f876b49f4af6b83d98cbbf4f8270/users/0dcc5314030048e78efd3405b291405f/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.375047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b589f876b49f4af6b83d98cbbf4f8270, user_id=0dcc5314030048e78efd3405b291405f, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.377676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.390762 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1254/2517] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:27 2022] PUT /identity/v3/projects/b589f876b49f4af6b83d98cbbf4f8270/users/0dcc5314030048e78efd3405b291405f/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:27.402383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.402672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.402978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.428805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] MFA Rules not processed for user `0dcc5314030048e78efd3405b291405f`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:27.430009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:27.493922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1264/2518] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:27.498911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.559636 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11d1acf-b85b-4ece-967d-48589690a870 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.564280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.564930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.565218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.565496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.566189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.568115 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.592638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1255/2519] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:27.596584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.607109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee1c1c-f70d-4500-ac20-e9381ecea334 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.632606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:27.664319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.664985 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.665271 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.665555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.666298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.668496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.699859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1265/2520] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:27.703828 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.713883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aaaf67-4c5c-407c-bbd1-6e802e3ec3b7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.718692 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.719380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.719681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.719953 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.720671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.722329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.733310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1256/2521] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:27.737548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.748301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30faf35-3f86-4bd4-8738-d4e4d1aa4db1 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.752940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.753813 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.754097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.754374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] PATH_INFO: `/v3/projects/ac0b6520c41245c0a27383734789e40b/users/d75ccb2348784478b3d7fd8ef59b3b04/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.769104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0b6520c41245c0a27383734789e40b, user_id=d75ccb2348784478b3d7fd8ef59b3b04, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.771791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.784070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1266/2522] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:27 2022] PUT /identity/v3/projects/ac0b6520c41245c0a27383734789e40b/users/d75ccb2348784478b3d7fd8ef59b3b04/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:27.787943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.798234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4ed29-4b7a-478b-90f4-3116a3d66aaa admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.826353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:27.855915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c755f20-e5b7-43d7-877f-cd038f5caac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.856575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755f20-e5b7-43d7-877f-cd038f5caac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.856860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755f20-e5b7-43d7-877f-cd038f5caac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.857140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c755f20-e5b7-43d7-877f-cd038f5caac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.857855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c755f20-e5b7-43d7-877f-cd038f5caac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.859735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c755f20-e5b7-43d7-877f-cd038f5caac4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.869246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1257/2523] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:25:27.873921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:27.884050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853527e1-b473-41ab-aa1f-6494bc1f5706 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:27.888662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:27.889523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.889805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.890077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] PATH_INFO: `/v3/projects/ac0b6520c41245c0a27383734789e40b/users/d75ccb2348784478b3d7fd8ef59b3b04/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.892041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac0b6520c41245c0a27383734789e40b, user_id=d75ccb2348784478b3d7fd8ef59b3b04, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:27.894652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:27.904719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1267/2524] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:27 2022] PUT /identity/v3/projects/ac0b6520c41245c0a27383734789e40b/users/d75ccb2348784478b3d7fd8ef59b3b04/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:27.914477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:27.914994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:27.915240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:27.941986 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] MFA Rules not processed for user `d75ccb2348784478b3d7fd8ef59b3b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:27.943247 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:28.010995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1258/2525] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:28.022256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:28.023970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:28.080752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d3e3-2e9b-40df-920d-46c423232647 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:28.084225 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:28.084850 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:28.085130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:28.085401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:28.095581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:28.097038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:28.107208 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1268/2526] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:28.175234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:28.184826 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff05619-9a2f-4242-9195-014d3f6b7ea5 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:28.188510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61865ab1-5dd5-4859-bc5c-14a0f59133ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:28.189145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61865ab1-5dd5-4859-bc5c-14a0f59133ce service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:28.189429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61865ab1-5dd5-4859-bc5c-14a0f59133ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:28.189706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61865ab1-5dd5-4859-bc5c-14a0f59133ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:28.245678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61865ab1-5dd5-4859-bc5c-14a0f59133ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:28.247132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61865ab1-5dd5-4859-bc5c-14a0f59133ce service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:28.256686 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1259/2527] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:28.512468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:28.575827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824d9a1-18b9-4b5c-9ba3-328aff109940 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:28.580331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:28.581187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:28.581466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:28.581737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] PATH_INFO: `/v3/users/0dcc5314030048e78efd3405b291405f` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:28.582934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dcc5314030048e78efd3405b291405f)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:28.584837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:28.612630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:28.630796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dcc5314030048e78efd3405b291405f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:28.631466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dcc5314030048e78efd3405b291405f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:28.639124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dcc5314030048e78efd3405b291405f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:28.639743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dcc5314030048e78efd3405b291405f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:28.651612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dcc5314030048e78efd3405b291405f'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:28.664301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1269/2528] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:28 2022] DELETE /identity/v3/users/0dcc5314030048e78efd3405b291405f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:28.668800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:28.669094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:28.669377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:28.686656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:28.687894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:28.734964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:28.749615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1260/2529] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:28.759304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:28.760705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:28.816931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934a3f8-4dff-4f6a-845e-1923d523da98 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:28.820567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:28.821198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:28.821479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:28.821752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:28.831246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:28.832652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:28.841991 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1270/2530] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:29.103965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:29.105446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:29.119246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8351c-6795-4cb6-acc6-b885704e3a6f None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:29.125084 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:29.126134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.126395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.126672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] PATH_INFO: `/v3/projects/b589f876b49f4af6b83d98cbbf4f8270` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.128029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b589f876b49f4af6b83d98cbbf4f8270)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:29.130011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:29.177797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b589f876b49f4af6b83d98cbbf4f8270', 'request_id': 'req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.184644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b589f876b49f4af6b83d98cbbf4f8270', 'request_id': 'req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.192028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b589f876b49f4af6b83d98cbbf4f8270', 'request_id': 'req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.202215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] The token cache is being invalidate because project b589f876b49f4af6b83d98cbbf4f8270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:29.202504 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.203138 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.204664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1261/2531] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:29 2022] DELETE /identity/v3/projects/b589f876b49f4af6b83d98cbbf4f8270 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:29.208595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:29.218273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf1780-2138-47a7-869b-77e1cbd591a7 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:29.274567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:29.275449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.275732 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.276009 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] PATH_INFO: `/v3/users/d75ccb2348784478b3d7fd8ef59b3b04` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.277190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75ccb2348784478b3d7fd8ef59b3b04)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:29.279132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:29.325875 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75ccb2348784478b3d7fd8ef59b3b04'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.326509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75ccb2348784478b3d7fd8ef59b3b04'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.327726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:29.335611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75ccb2348784478b3d7fd8ef59b3b04'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.336335 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75ccb2348784478b3d7fd8ef59b3b04'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.348393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ea964417-e1b7-49ac-811b-5bcf90e837dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75ccb2348784478b3d7fd8ef59b3b04'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.361656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1271/2532] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:29 2022] DELETE /identity/v3/users/d75ccb2348784478b3d7fd8ef59b3b04 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:29.627529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:29.628994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:29.642522 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6878a5-adb5-4ba5-9b0c-ab19103b91ed admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:29.698703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:29.699630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.699913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.700192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] PATH_INFO: `/v3/projects/ac0b6520c41245c0a27383734789e40b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.701366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac0b6520c41245c0a27383734789e40b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:29.703314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:29.749237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac0b6520c41245c0a27383734789e40b', 'request_id': 'req-8cde7f67-ae80-4bf9-844c-52c9b429b950'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.755259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac0b6520c41245c0a27383734789e40b', 'request_id': 'req-8cde7f67-ae80-4bf9-844c-52c9b429b950'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.762537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac0b6520c41245c0a27383734789e40b', 'request_id': 'req-8cde7f67-ae80-4bf9-844c-52c9b429b950'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.772589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] The token cache is being invalidate because project ac0b6520c41245c0a27383734789e40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:29.772877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.773488 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8cde7f67-ae80-4bf9-844c-52c9b429b950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:29.775061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1262/2533] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:29 2022] DELETE /identity/v3/projects/ac0b6520c41245c0a27383734789e40b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:29.780400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.780690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.780967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.799337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:29.800540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:29.857995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1272/2534] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:29.867191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.867550 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.867836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.875100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:29.888100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:29.889302 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:29.897196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1263/2535] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:29.901275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:29.911538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e6374-3327-4112-9df9-53ae1bd29abb None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:29.916188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:29.916845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.917132 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.918035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.918771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:29.919115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:29.920701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:29.932159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1273/2536] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:25:29.936228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:29.945204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f50b73-e16e-4188-981c-c974204cbafa None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:29.950483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:29.951155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:29.951443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:29.951720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:29.952419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:29.954306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:29.979990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1264/2537] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:29 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:25:29.983898 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:29.994059 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c207566-a1b6-4b64-9026-abe2daf16f41 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.049184 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.049782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.050070 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.050377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.051083 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.053007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.072100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1274/2538] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:29 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:30.076578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.086799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6946143-124c-4c5d-b35d-04f824b3526f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.092139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.092787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.093077 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.093356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.094080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.095705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.105924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1265/2539] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:30.109802 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.119497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f517b6-f71b-4c9a-8440-a72a53b045c6 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.124026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5a3f6-6cc5-4095-92b8-63890003e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.124666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a3f6-6cc5-4095-92b8-63890003e508 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.124944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a3f6-6cc5-4095-92b8-63890003e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.125215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5a3f6-6cc5-4095-92b8-63890003e508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.125931 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a3f6-6cc5-4095-92b8-63890003e508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.127623 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b5a3f6-6cc5-4095-92b8-63890003e508 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.137754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1275/2540] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:30.141515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.151660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5490d-cfba-44ff-9500-adc95bb9f9d6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.156823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.157705 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.157990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.158268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] PATH_INFO: `/v3/projects/d14a858ea4fe4060b3f23aa5935c3128/users/a3cd76f57f8d41ee93a24e6a4b772c89/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.172741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14a858ea4fe4060b3f23aa5935c3128, user_id=a3cd76f57f8d41ee93a24e6a4b772c89, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.177213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.193307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1266/2541] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:30 2022] PUT /identity/v3/projects/d14a858ea4fe4060b3f23aa5935c3128/users/a3cd76f57f8d41ee93a24e6a4b772c89/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:30.204173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.204463 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.204741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.231082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] MFA Rules not processed for user `a3cd76f57f8d41ee93a24e6a4b772c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:30.232254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:30.250356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:30.306684 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1276/2542] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:30.311781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.372573 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40c7ea-1a81-4a39-9674-341af9d4abc9 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.377412 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.378062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.378350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.378629 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.379397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.381309 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.406046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1267/2543] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:30 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:30.410005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.420760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99927cc3-0526-4240-855e-66c6cea12b9b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.485131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.485796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.486076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.486350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.487062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.488995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.508389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1277/2544] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:30 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:30.512928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.523250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076e64f-684d-4948-a1a1-36cbe1414dad admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.527965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.528618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.528907 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.529187 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.529948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.531590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.542042 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1268/2545] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:30.546019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.556261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710113a3-c385-4b7c-aecc-e4271e1d9553 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.561561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.562371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.562655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.562964 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] PATH_INFO: `/v3/projects/7ea870176e4945a8b58cf74d01ebf5c8/users/1e1dd2d56d284a639ffb18699b289129/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.579670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea870176e4945a8b58cf74d01ebf5c8, user_id=1e1dd2d56d284a639ffb18699b289129, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.582319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.595012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1278/2546] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:30 2022] PUT /identity/v3/projects/7ea870176e4945a8b58cf74d01ebf5c8/users/1e1dd2d56d284a639ffb18699b289129/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:30.599160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.609314 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bcf83-207a-4219-806c-20ce9100c583 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.669290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79a553-5952-4039-819c-3a0b62c36292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.669962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79a553-5952-4039-819c-3a0b62c36292 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.670252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79a553-5952-4039-819c-3a0b62c36292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.670537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79a553-5952-4039-819c-3a0b62c36292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.671308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79a553-5952-4039-819c-3a0b62c36292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.673250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79a553-5952-4039-819c-3a0b62c36292 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.684137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1269/2547] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:30.688385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.699500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d613c8-d17a-4d24-9019-9753e53cd25e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.704242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.705205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.705491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.705763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] PATH_INFO: `/v3/projects/7ea870176e4945a8b58cf74d01ebf5c8/users/1e1dd2d56d284a639ffb18699b289129/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.707775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea870176e4945a8b58cf74d01ebf5c8, user_id=1e1dd2d56d284a639ffb18699b289129, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.710399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:30.727008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1279/2548] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:30 2022] PUT /identity/v3/projects/7ea870176e4945a8b58cf74d01ebf5c8/users/1e1dd2d56d284a639ffb18699b289129/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:30.738864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.739160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.739440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.759225 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:30.770707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] MFA Rules not processed for user `1e1dd2d56d284a639ffb18699b289129`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:30.772082 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:30.851193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1270/2549] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:30.863024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:30.864906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:30.929632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576ae4d-8ae7-491d-a4ce-fb54a1e14356 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:30.933831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a657a3e-19b3-4c0f-9e06-ec7c998f0ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:30.934469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a657a3e-19b3-4c0f-9e06-ec7c998f0ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:30.934801 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a657a3e-19b3-4c0f-9e06-ec7c998f0ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:30.935199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a657a3e-19b3-4c0f-9e06-ec7c998f0ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:30.963884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:30.997077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a657a3e-19b3-4c0f-9e06-ec7c998f0ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:30.998567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a657a3e-19b3-4c0f-9e06-ec7c998f0ce8 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:31.007546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1280/2550] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:33.513704 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:33.607051 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf626cd-129e-4c78-b5d7-be5ec5153cda None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:33.612473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:33.613514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:33.613838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:33.614135 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] PATH_INFO: `/v3/users/a3cd76f57f8d41ee93a24e6a4b772c89` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:33.615711 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3cd76f57f8d41ee93a24e6a4b772c89)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:33.618029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:33.687164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3cd76f57f8d41ee93a24e6a4b772c89'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:33.688535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3cd76f57f8d41ee93a24e6a4b772c89'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:33.699043 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3cd76f57f8d41ee93a24e6a4b772c89'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:33.699808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3cd76f57f8d41ee93a24e6a4b772c89'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:33.708414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3cd76f57f8d41ee93a24e6a4b772c89'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:33.727761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1271/2551] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:33 2022] DELETE /identity/v3/users/a3cd76f57f8d41ee93a24e6a4b772c89 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:33.732533 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:33.733001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:33.733098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:33.752981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:33.754220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:33.813927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1281/2552] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:33.823246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:33.824669 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:33.843308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:33.886871 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45539939-f0f7-40f4-8cf0-20c4577095b3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:33.890820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:33.891571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:33.892036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:33.892327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:33.901612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:33.903066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:33.912751 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1272/2553] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:34.197082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:34.198729 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:34.211494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337ec4-ecf2-4e95-a7d6-d422607d8c61 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:34.216764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:34.217641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:34.217934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:34.218204 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] PATH_INFO: `/v3/projects/d14a858ea4fe4060b3f23aa5935c3128` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:34.219469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14a858ea4fe4060b3f23aa5935c3128)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:34.221606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:34.273126 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14a858ea4fe4060b3f23aa5935c3128', 'request_id': 'req-f5162874-9b53-4815-9990-deec63150e09'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.283134 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14a858ea4fe4060b3f23aa5935c3128', 'request_id': 'req-f5162874-9b53-4815-9990-deec63150e09'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.289328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14a858ea4fe4060b3f23aa5935c3128', 'request_id': 'req-f5162874-9b53-4815-9990-deec63150e09'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.301177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] The token cache is being invalidate because project d14a858ea4fe4060b3f23aa5935c3128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:34.301468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.302121 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.303716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1282/2554] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:34 2022] DELETE /identity/v3/projects/d14a858ea4fe4060b3f23aa5935c3128 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:34.307912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:34.317744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffdd67-2ab6-402e-acfa-e31ed5dde4da service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:34.405741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:34.406632 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:34.407031 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:34.407286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] PATH_INFO: `/v3/users/1e1dd2d56d284a639ffb18699b289129` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:34.408561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1dd2d56d284a639ffb18699b289129)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:34.410708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:34.478164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1dd2d56d284a639ffb18699b289129'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.478920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1dd2d56d284a639ffb18699b289129'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.488429 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1dd2d56d284a639ffb18699b289129'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.489250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1dd2d56d284a639ffb18699b289129'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.498873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8d4fa9d9-258d-4e13-95b7-30ec05e8fded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1dd2d56d284a639ffb18699b289129'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:34.518993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1273/2555] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:34 2022] DELETE /identity/v3/users/1e1dd2d56d284a639ffb18699b289129 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:34.853656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:34.855831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:34.877212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5162874-9b53-4815-9990-deec63150e09 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:34.929253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:34.940194 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:34.941473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:34.942097 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:34.942907 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] PATH_INFO: `/v3/projects/7ea870176e4945a8b58cf74d01ebf5c8` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:34.944348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea870176e4945a8b58cf74d01ebf5c8)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:34.946355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:34.995341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea870176e4945a8b58cf74d01ebf5c8', 'request_id': 'req-30e4b4d8-5639-4787-8144-a5dbe6998daf'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.004017 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea870176e4945a8b58cf74d01ebf5c8', 'request_id': 'req-30e4b4d8-5639-4787-8144-a5dbe6998daf'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.010423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea870176e4945a8b58cf74d01ebf5c8', 'request_id': 'req-30e4b4d8-5639-4787-8144-a5dbe6998daf'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.028010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] The token cache is being invalidate because project 7ea870176e4945a8b58cf74d01ebf5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:35.028309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.028949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-30e4b4d8-5639-4787-8144-a5dbe6998daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.030517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1283/2556] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:34 2022] DELETE /identity/v3/projects/7ea870176e4945a8b58cf74d01ebf5c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:35.035396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.035691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.035974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.056010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:35.057268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:35.096121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:35.126784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1274/2557] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:35.134649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a235245-77ea-4719-ba82-e516e547728d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.135053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a235245-77ea-4719-ba82-e516e547728d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.135357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a235245-77ea-4719-ba82-e516e547728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.158464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4a235245-77ea-4719-ba82-e516e547728d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:35.159828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4a235245-77ea-4719-ba82-e516e547728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:35.169969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1284/2558] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:35.174274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.183934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b96eb0-b63a-47c2-b5e6-222ecd9e6af3 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.189069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.189850 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.190237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.190613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.191487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:35.191894 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.193597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.206032 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1275/2559] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:25:35.210163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a235245-77ea-4719-ba82-e516e547728d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.220018 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a235245-77ea-4719-ba82-e516e547728d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.224661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.225298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.225582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.225853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.226559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.228479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.252949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1285/2560] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:35 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:35.257091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.267387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21866f-7938-47aa-8d5b-538b153e5b72 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.324335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.324991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.325274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.325549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.326232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.328185 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.348680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1276/2561] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:35 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:35.352976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.363698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17d90b-cab9-4c12-98b2-d26c204e97ef admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.369156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.369807 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.370094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.370367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.371322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.373362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.385485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1286/2562] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:25:35.390076 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.400902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf08c98-8962-422e-a2a4-f63e24530283 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.406820 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea59fd1-fb85-404b-b622-d2e9ce8621aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.407755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea59fd1-fb85-404b-b622-d2e9ce8621aa admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.408040 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea59fd1-fb85-404b-b622-d2e9ce8621aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.408334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea59fd1-fb85-404b-b622-d2e9ce8621aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.409348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea59fd1-fb85-404b-b622-d2e9ce8621aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.411011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea59fd1-fb85-404b-b622-d2e9ce8621aa admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.425140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1277/2563] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:35.426386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.436547 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60785b4-f8a9-4a72-b771-02191dff6430 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.441199 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.442069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.442351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.442625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] PATH_INFO: `/v3/projects/15e45492213647c3b2779bbb0ca317e7/users/794ae620eaa74bb6a0afca335e79f1fa/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.457858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e45492213647c3b2779bbb0ca317e7, user_id=794ae620eaa74bb6a0afca335e79f1fa, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.460608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.474227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1287/2564] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:35 2022] PUT /identity/v3/projects/15e45492213647c3b2779bbb0ca317e7/users/794ae620eaa74bb6a0afca335e79f1fa/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:35.484323 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.484615 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.485146 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.510519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] MFA Rules not processed for user `794ae620eaa74bb6a0afca335e79f1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:35.512218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:35.576643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1278/2565] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:35.587816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.589506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:35.645257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c881a9b-722b-434a-9301-3c40aec58beb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.648912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19487f31-4810-4a96-828f-ec8bbc2eb519 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.649539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19487f31-4810-4a96-828f-ec8bbc2eb519 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.649823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19487f31-4810-4a96-828f-ec8bbc2eb519 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.650101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19487f31-4810-4a96-828f-ec8bbc2eb519 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.658348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19487f31-4810-4a96-828f-ec8bbc2eb519 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.659803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19487f31-4810-4a96-828f-ec8bbc2eb519 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.668611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1288/2566] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:35.692454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.725735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:35.754643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99f28c-75e6-4fa2-8f91-09dcb812aa21 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:35.759239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:35.760090 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.760371 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.760643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] PATH_INFO: `/v3/users/794ae620eaa74bb6a0afca335e79f1fa` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.762415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=794ae620eaa74bb6a0afca335e79f1fa)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:35.764378 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:35.809049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '794ae620eaa74bb6a0afca335e79f1fa'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.809710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '794ae620eaa74bb6a0afca335e79f1fa'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.816590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '794ae620eaa74bb6a0afca335e79f1fa'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.817228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '794ae620eaa74bb6a0afca335e79f1fa'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.828491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '794ae620eaa74bb6a0afca335e79f1fa'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:35.846485 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1279/2567] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:35 2022] DELETE /identity/v3/users/794ae620eaa74bb6a0afca335e79f1fa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:35.851165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:35.851457 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:35.851734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:35.869242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:35.870433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:35.882892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:35.929780 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1289/2568] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:35.939318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:35.940667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:35.996507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b77a06-e61b-4e3f-93a5-049c51f939c6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.000150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ef9c-dfd7-4978-8052-106dba9e90b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.000788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ef9c-dfd7-4978-8052-106dba9e90b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.001070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ef9c-dfd7-4978-8052-106dba9e90b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.001345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ef9c-dfd7-4978-8052-106dba9e90b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.010132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3ef9c-dfd7-4978-8052-106dba9e90b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.012079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3ef9c-dfd7-4978-8052-106dba9e90b1 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.021244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1280/2569] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:36.302945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.304541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:36.319851 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d567df-4a92-4b80-be41-e516c2b908fb None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.325441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.326953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.327163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.327440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] PATH_INFO: `/v3/projects/15e45492213647c3b2779bbb0ca317e7` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.328861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e45492213647c3b2779bbb0ca317e7)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.330925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.381473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e45492213647c3b2779bbb0ca317e7', 'request_id': 'req-1d638cc7-d5f2-4af4-84d2-8abdad88482a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:36.391065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e45492213647c3b2779bbb0ca317e7', 'request_id': 'req-1d638cc7-d5f2-4af4-84d2-8abdad88482a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:36.396935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e45492213647c3b2779bbb0ca317e7', 'request_id': 'req-1d638cc7-d5f2-4af4-84d2-8abdad88482a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:36.415890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] The token cache is being invalidate because project 15e45492213647c3b2779bbb0ca317e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:36.416181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:36.416823 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1d638cc7-d5f2-4af4-84d2-8abdad88482a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:36.418415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1290/2570] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:36 2022] DELETE /identity/v3/projects/15e45492213647c3b2779bbb0ca317e7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:36.423278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.423570 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.423846 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.441904 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:36.443145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:36.492173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:36.505470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1281/2571] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:36.515196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.515488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.515768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.535507 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:36.536765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:36.545217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1291/2572] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:36.549682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.560308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3fd0c-d41b-43ce-87d4-754b30e0ac03 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.565191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.565839 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.566122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.566394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.567158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:36.567489 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.569152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.581637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1282/2573] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:25:36.586328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.596275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2addf20-9d97-4db8-a1da-c2d405e2aace None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.601150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.601856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.602139 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.602413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.603176 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.605050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.631004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1292/2574] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:36 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:36.635140 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.645939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa61dd-a2df-4178-a2b1-6a7128cac707 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.700932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.701625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.701911 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.702189 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.702915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.704829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.723715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1283/2575] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:36 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:25:36.727972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.738890 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78a145-4e34-4742-a1a5-e8f22330c92f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.743482 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.744066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.744346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.744618 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.745428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.747796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.757869 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1293/2576] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:36.761641 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.771538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af635e5d-7df3-472c-b6cf-da4cf2ff1786 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.776121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f26ab-1126-496d-b1b1-797c55c30d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.776764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f26ab-1126-496d-b1b1-797c55c30d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.777054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f26ab-1126-496d-b1b1-797c55c30d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.777327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f26ab-1126-496d-b1b1-797c55c30d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.778041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f26ab-1126-496d-b1b1-797c55c30d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.779661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f26ab-1126-496d-b1b1-797c55c30d53 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.789432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1284/2577] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:36.793749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:36.803508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0020-0786-4f76-aa31-1582d7a901e2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:36.808660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:36.809529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.809839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.810219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] PATH_INFO: `/v3/projects/7db32e2da53d4a9c80ab1d47452de998/users/c8ddb92381e148b8bdbcc05cfa9cec49/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.824114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db32e2da53d4a9c80ab1d47452de998, user_id=c8ddb92381e148b8bdbcc05cfa9cec49, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:36.826717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:36.839275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1294/2578] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:36 2022] PUT /identity/v3/projects/7db32e2da53d4a9c80ab1d47452de998/users/c8ddb92381e148b8bdbcc05cfa9cec49/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:36.901351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:36.901642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:36.901970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:36.928607 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] MFA Rules not processed for user `c8ddb92381e148b8bdbcc05cfa9cec49`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:36.929877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:36.992279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1285/2579] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:37.003822 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:37.005518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:37.048790 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:37.080066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd903b-30dc-4c00-8af6-df16a43521cc admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:37.086306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:37.087082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:37.087371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:37.087650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:37.104697 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:37.110683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:37.125007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1295/2580] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:37.513326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:37.514733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:37.571252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba664d8-be08-44d4-ad35-fafa4dac541a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:37.574732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:37.575396 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:37.575683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:37.575958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:37.577681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:37.582426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:37.640954 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:37.642376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:37.651956 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1286/2581] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:40.504685 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:40.506124 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:40.520192 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35d541-482e-4803-9485-9bce367efe56 service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:40.524756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:40.525615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:40.525894 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:40.526167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] PATH_INFO: `/v3/users/326e0d70244f46059664a135aefab0f0` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:40.527392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326e0d70244f46059664a135aefab0f0)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:40.529290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:40.577638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326e0d70244f46059664a135aefab0f0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.578292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326e0d70244f46059664a135aefab0f0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.585377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326e0d70244f46059664a135aefab0f0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.586035 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326e0d70244f46059664a135aefab0f0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.599286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d13d9990-8c69-44a8-b59f-e7e1deae6d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326e0d70244f46059664a135aefab0f0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.612962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1296/2582] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:40 2022] DELETE /identity/v3/users/326e0d70244f46059664a135aefab0f0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:40.790106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:40.800749 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06288-08f0-4c4c-bff2-38be37b6b9c9 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:40.863331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:40.864182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:40.864464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:40.864737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] PATH_INFO: `/v3/projects/346889f9c6334aa9a22bd277aa002a0f` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:40.865926 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=346889f9c6334aa9a22bd277aa002a0f)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:40.867876 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:40.911932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346889f9c6334aa9a22bd277aa002a0f', 'request_id': 'req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.918592 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346889f9c6334aa9a22bd277aa002a0f', 'request_id': 'req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.925816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346889f9c6334aa9a22bd277aa002a0f', 'request_id': 'req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.936120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] The token cache is being invalidate because project 346889f9c6334aa9a22bd277aa002a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:40.936408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.937033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-53e2c14e-57cc-463c-ad55-fbd3e4c906a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:40.938548 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1287/2583] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:40 2022] DELETE /identity/v3/projects/346889f9c6334aa9a22bd277aa002a0f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:40.943492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:40.943762 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:40.944040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:40.961143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:40.962504 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:41.020674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1297/2584] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:41.028584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.028905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.029204 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.047708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:41.048905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:41.056732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1288/2585] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:41.060937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.070490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f4493-0c12-46ad-bbf9-4b25bc79ddcf None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.075125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.075798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.076076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.076346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.077071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:41.077373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.078990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.091216 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1298/2586] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:25:41.095985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.105973 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f411c-045f-47bc-adb3-d56b4a1f9dbe None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.111001 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.111648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.111937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.112219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.112929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.114858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.143501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1289/2587] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:41 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:41.144600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.155209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b9b9-1977-4d0a-9315-a45fd33191f4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.194552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:41.212723 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.213318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.213603 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.213881 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.215171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.217080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.236942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1299/2588] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:41 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:25:41.241112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.246498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:41.252808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cf46b-6467-4075-89a3-e0d8b365717b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.258364 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.259145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.259435 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.259716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.260446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.262535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.272874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1290/2589] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:41.276651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.287205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f29026-ea53-401f-a45f-29ab196e6516 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.291858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57618ddc-8540-49ed-a4a6-2a265723ad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.292535 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618ddc-8540-49ed-a4a6-2a265723ad33 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.292820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618ddc-8540-49ed-a4a6-2a265723ad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.293098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57618ddc-8540-49ed-a4a6-2a265723ad33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.293825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618ddc-8540-49ed-a4a6-2a265723ad33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.295485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57618ddc-8540-49ed-a4a6-2a265723ad33 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.305215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1300/2590] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:41.309761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.319724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9a61-0294-4db7-a0a7-1ed03c279dc9 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.324851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.325713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.326188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.326291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] PATH_INFO: `/v3/projects/d0f55b1f83ce4fa7b76f996286bd0330/users/8462e1b2ad9b426b91ffde570f3c2c1c/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.341030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f55b1f83ce4fa7b76f996286bd0330, user_id=8462e1b2ad9b426b91ffde570f3c2c1c, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.343727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.355965 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1291/2591] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:41 2022] PUT /identity/v3/projects/d0f55b1f83ce4fa7b76f996286bd0330/users/8462e1b2ad9b426b91ffde570f3c2c1c/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:41.367202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.367515 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.367812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.393768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] MFA Rules not processed for user `8462e1b2ad9b426b91ffde570f3c2c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:41.395001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:41.459236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1301/2592] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:41.470471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:41.472201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:41.528413 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f875662-5ae7-444d-a852-18555d3c4e5f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:41.532124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c7ae5e-450c-436c-ac3b-852b91c2539d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:41.532768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7ae5e-450c-436c-ac3b-852b91c2539d service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:41.533055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7ae5e-450c-436c-ac3b-852b91c2539d service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:41.533337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c7ae5e-450c-436c-ac3b-852b91c2539d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:41.541589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7ae5e-450c-436c-ac3b-852b91c2539d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:41.543655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c7ae5e-450c-436c-ac3b-852b91c2539d service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:41.552376 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1292/2593] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:44.921450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:44.931318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e8319-9997-4ccd-98fc-9ad7638e4929 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:44.934969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:44.935602 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:44.935882 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:44.936154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:44.937920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:44.996546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:44.997975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:45.007158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1302/2594] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:51.548474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:55.002968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:55.999836 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b270a-2ca4-488f-bdb1-01ae99d98f0b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:56.000345 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b270a-2ca4-488f-bdb1-01ae99d98f0b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:56.000453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b270a-2ca4-488f-bdb1-01ae99d98f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:56.017035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1c1b270a-2ca4-488f-bdb1-01ae99d98f0b None None] MFA Rules not processed for user `36c5458a18f146be87207e0f6dd598eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:56.018211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1c1b270a-2ca4-488f-bdb1-01ae99d98f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:56.024985 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1293/2595] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:58.435702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:58.438079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:58.516421 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecee9c-a083-4417-897c-07816b24abef service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:58.521080 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:58.521951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:58.522235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:58.522508 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] PATH_INFO: `/v3/users/c8ddb92381e148b8bdbcc05cfa9cec49` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:58.523887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ddb92381e148b8bdbcc05cfa9cec49)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:58.525934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:58.579703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ddb92381e148b8bdbcc05cfa9cec49'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:58.580375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ddb92381e148b8bdbcc05cfa9cec49'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:58.587472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ddb92381e148b8bdbcc05cfa9cec49'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:58.588112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ddb92381e148b8bdbcc05cfa9cec49'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:58.602262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ddb92381e148b8bdbcc05cfa9cec49'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:58.616734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1303/2596] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:58 2022] DELETE /identity/v3/users/c8ddb92381e148b8bdbcc05cfa9cec49 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:58.621265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:58.621781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:58.622362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:58.641012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:58.642250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:58.726905 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1294/2597] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:25:58.736903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:58.738436 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:58.823690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315eae54-e1b6-4cca-9a5b-f9350b6e75d0 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:58.828278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:58.828951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:58.829231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:58.829502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:58.839419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:58.841393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:58.853581 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1304/2598] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:59.285222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.286957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:59.300932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.301075 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65fdde-73ed-4981-bbbe-3fbe052e1ca7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.302393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:59.307811 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.308707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.309165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.309544 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] PATH_INFO: `/v3/projects/7db32e2da53d4a9c80ab1d47452de998` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.310946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db32e2da53d4a9c80ab1d47452de998)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.313250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.330072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea3baa-259a-4a41-bcb2-f0baeb4747cc service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.339856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.341606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.342053 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.342476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] PATH_INFO: `/v3/users/8462e1b2ad9b426b91ffde570f3c2c1c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.345158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8462e1b2ad9b426b91ffde570f3c2c1c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.347301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.381179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:59.385201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db32e2da53d4a9c80ab1d47452de998', 'request_id': 'req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.395143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db32e2da53d4a9c80ab1d47452de998', 'request_id': 'req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.409393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db32e2da53d4a9c80ab1d47452de998', 'request_id': 'req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.431324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8462e1b2ad9b426b91ffde570f3c2c1c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.432015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8462e1b2ad9b426b91ffde570f3c2c1c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.437073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] The token cache is being invalidate because project 7db32e2da53d4a9c80ab1d47452de998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:25:59.437372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.438047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-81d5c63b-56a5-4663-a3fb-aefe5c3ce6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.440018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1295/2599] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:25:59 2022] DELETE /identity/v3/projects/7db32e2da53d4a9c80ab1d47452de998 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:25:59.446411 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8462e1b2ad9b426b91ffde570f3c2c1c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.446981 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23720-9ba2-40f6-adf4-901e2dd8f454 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.447198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23720-9ba2-40f6-adf4-901e2dd8f454 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.447410 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e23720-9ba2-40f6-adf4-901e2dd8f454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.447516 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8462e1b2ad9b426b91ffde570f3c2c1c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.470377 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7707f315-6052-4f94-b469-72ae65f034ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8462e1b2ad9b426b91ffde570f3c2c1c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:25:59.473547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d5e23720-9ba2-40f6-adf4-901e2dd8f454 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:59.474697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d5e23720-9ba2-40f6-adf4-901e2dd8f454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:59.496539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1305/2600] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:25:59 2022] DELETE /identity/v3/users/8462e1b2ad9b426b91ffde570f3c2c1c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:25:59.499293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.499953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.500511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.521246 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:59.522484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:59.552599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1296/2601] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:59.560834 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.561389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.562125 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.591802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:25:59.593449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:59.620256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:59.646711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1306/2602] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:59.657661 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.661107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:25:59.682596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1297/2603] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:25:59.687507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.701491 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232099b-b33f-4e5e-b9fd-a7417d10eb5e None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.707213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.708232 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.708738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.709532 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.710509 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:25:59.711070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.712901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.729139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1298/2604] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:25:59.733441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.740942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f222f54-de47-40e6-8cb1-b5177b65c257 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.744744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.745398 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.745681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.745802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dcc0a-375b-48e5-8b5d-497443576364 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.745998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.751443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.752107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.752392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.752831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.753535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.756526 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.757021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.758453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.770550 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1307/2605] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:25:59.789672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1299/2606] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:25:59 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:25:59.790863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.802273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36c62f-7204-4a3a-9949-0799ab617d3f service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.842115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:25:59.876979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.877656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.877942 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.878218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.879081 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.881000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.906013 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1308/2607] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:25:59 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:25:59.907192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.917648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dac91-b74a-4405-be5b-c591ae50338f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.923531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.924243 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.924477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.924754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.925479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.927108 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.941725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1300/2608] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:59.943415 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.957464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dcec3-651a-42b4-85bf-98bb8f03dc01 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:25:59.963318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9de87e-fbcd-41fd-a1b0-efd5531c0e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:25:59.964157 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9de87e-fbcd-41fd-a1b0-efd5531c0e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:25:59.964450 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9de87e-fbcd-41fd-a1b0-efd5531c0e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:25:59.964803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9de87e-fbcd-41fd-a1b0-efd5531c0e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:25:59.965703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9de87e-fbcd-41fd-a1b0-efd5531c0e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:25:59.967473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9de87e-fbcd-41fd-a1b0-efd5531c0e32 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:25:59.982978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1309/2609] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:25:59.984757 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:25:59.997929 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6305f-7bf6-4860-98ee-d27167c65ee8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.003754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.004853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.005764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.006256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] PATH_INFO: `/v3/projects/6f612ecb68c2414f92e3887ffeff0cdd/users/0c13da70175548fbbddc42879c3024d5/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.023718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f612ecb68c2414f92e3887ffeff0cdd, user_id=0c13da70175548fbbddc42879c3024d5, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.027003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.049209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1301/2610] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:25:59 2022] PUT /identity/v3/projects/6f612ecb68c2414f92e3887ffeff0cdd/users/0c13da70175548fbbddc42879c3024d5/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:00.051045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.051569 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.051913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.072749 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:00.073959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:00.097598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.099768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:00.157142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1310/2611] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:00.166397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.192931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc56a3-10d5-4a3c-93e5-5ec4dc388e58 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.197578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.198448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.198734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.199046 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] PATH_INFO: `/v3/projects/d0f55b1f83ce4fa7b76f996286bd0330` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.200761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f55b1f83ce4fa7b76f996286bd0330)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.202681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.232785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1718e-6f9b-48ae-a4ed-1000d4340db0 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.237672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe904d8a-610e-465b-9da5-4f2eb337bc5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.238314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe904d8a-610e-465b-9da5-4f2eb337bc5a service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.238592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe904d8a-610e-465b-9da5-4f2eb337bc5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.238885 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe904d8a-610e-465b-9da5-4f2eb337bc5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.248781 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe904d8a-610e-465b-9da5-4f2eb337bc5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.250677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe904d8a-610e-465b-9da5-4f2eb337bc5a service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.253439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f55b1f83ce4fa7b76f996286bd0330', 'request_id': 'req-0181a13b-996c-4aa8-b483-6b1274a869ee'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:00.256049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:00.262145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f55b1f83ce4fa7b76f996286bd0330', 'request_id': 'req-0181a13b-996c-4aa8-b483-6b1274a869ee'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:00.262880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1311/2612] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:26:00.272503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f55b1f83ce4fa7b76f996286bd0330', 'request_id': 'req-0181a13b-996c-4aa8-b483-6b1274a869ee'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:00.284055 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] The token cache is being invalidate because project d0f55b1f83ce4fa7b76f996286bd0330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:00.284436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:00.285165 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-0181a13b-996c-4aa8-b483-6b1274a869ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:00.286721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1302/2613] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:00 2022] DELETE /identity/v3/projects/d0f55b1f83ce4fa7b76f996286bd0330 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:00.291966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.292256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.292532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.313285 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:00.314932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:00.374738 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:00.388977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1312/2614] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:00.396897 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.397186 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.397465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.416217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:00.417793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:00.428594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1303/2615] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:00.432940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.443261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2f35b-50fa-4c9a-9438-f91afea004b9 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.448796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.449528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.450355 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.450745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.451565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:00.451872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.453489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.466897 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1313/2616] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:00.479859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.497726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0c50a-4bdc-45bb-886e-41881c062143 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.504618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.505303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.505576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.505951 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.506535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.508567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.539028 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1304/2617] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:00.542774 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.556649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d122a1e-616e-4698-b65b-c8e0f9b4d4d2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.643166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.643849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.644131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.644407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.645299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.648066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.676946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1314/2618] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/users => generated 302 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:00.680699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.691682 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac6517-1c38-4afe-849f-1aca3ce97cf8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.696810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.697398 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.697681 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.697955 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.698675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.700333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.712586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1305/2619] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:00.716489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.726429 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f43da5-0856-44eb-88f6-feb28efbf682 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.731599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf1f5b-84a5-461b-a5f1-2883a98a5e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.732249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf1f5b-84a5-461b-a5f1-2883a98a5e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.732530 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf1f5b-84a5-461b-a5f1-2883a98a5e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.732804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf1f5b-84a5-461b-a5f1-2883a98a5e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.733766 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf1f5b-84a5-461b-a5f1-2883a98a5e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.735467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf1f5b-84a5-461b-a5f1-2883a98a5e65 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.746589 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1315/2620] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:00.751183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.761199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a495c00-35e0-410a-b348-c4d2fd7e6f7a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.765879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.766756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.767065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.767337 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] PATH_INFO: `/v3/projects/26f9cfb95ef647f5a87c22ec4048d26c/users/231a011bff304db6bc4daa1594bec2af/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.783566 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f9cfb95ef647f5a87c22ec4048d26c, user_id=231a011bff304db6bc4daa1594bec2af, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.786267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:00.801116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1306/2621] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:00 2022] PUT /identity/v3/projects/26f9cfb95ef647f5a87c22ec4048d26c/users/231a011bff304db6bc4daa1594bec2af/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:00.812196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.812486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.812763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.849196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] MFA Rules not processed for user `231a011bff304db6bc4daa1594bec2af`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:00.850425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:00.918000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1316/2622] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:00.922853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:00.988226 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5224af-73f7-437b-8a01-ed763dcd4a4c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:00.993568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:00.994223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:00.994604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:00.995057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:00.995501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:00.997992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.024359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1307/2623] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:00 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:01.029037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.039440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c395a8c-0090-45cc-a681-bebd2fd5968a None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.091170 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:01.100281 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.100959 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.101239 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.101512 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.102210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.104207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.128005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1317/2624] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:01 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:01.129303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.139771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853130-7523-430c-acac-70d67994d2e8 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.147395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.148268 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.148559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.148833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.149559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.151535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.167062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1308/2625] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:01.168362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.180179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7935a-8da8-4645-b9a3-d9aa532c289f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.185828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.186796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.187485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.187485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] PATH_INFO: `/v3/projects/ed3256b8f89445c08f4b0fd4d448d4b1/users/2be0d0e6b7824583a6f4394d1109664b/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.239734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed3256b8f89445c08f4b0fd4d448d4b1, user_id=2be0d0e6b7824583a6f4394d1109664b, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.243381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.264666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1318/2626] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:01 2022] PUT /identity/v3/projects/ed3256b8f89445c08f4b0fd4d448d4b1/users/2be0d0e6b7824583a6f4394d1109664b/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:01.268922 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.280608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f6cd2-5f36-417b-8fd4-70cf3a096e48 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.335210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:01.355469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b323b0-b4e4-44b8-adb3-83ab1a4b0a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.356791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b323b0-b4e4-44b8-adb3-83ab1a4b0a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.357020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b323b0-b4e4-44b8-adb3-83ab1a4b0a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.357608 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b323b0-b4e4-44b8-adb3-83ab1a4b0a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.358891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b323b0-b4e4-44b8-adb3-83ab1a4b0a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.361157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b323b0-b4e4-44b8-adb3-83ab1a4b0a67 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.377096 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1309/2627] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:01.378348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.392485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039162-1c5e-49d7-af48-9acf138421a8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.398937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.399970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.400322 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.400677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] PATH_INFO: `/v3/projects/ed3256b8f89445c08f4b0fd4d448d4b1/users/2be0d0e6b7824583a6f4394d1109664b/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.402859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed3256b8f89445c08f4b0fd4d448d4b1, user_id=2be0d0e6b7824583a6f4394d1109664b, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.405787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.430874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1319/2628] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:01 2022] PUT /identity/v3/projects/ed3256b8f89445c08f4b0fd4d448d4b1/users/2be0d0e6b7824583a6f4394d1109664b/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:01.442572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.442888 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.443166 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.475645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] MFA Rules not processed for user `2be0d0e6b7824583a6f4394d1109664b`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:01.475896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:01.565578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1310/2629] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:01.573686 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.577156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:01.642783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603b6d0-cda0-4298-9b63-9cf70f6ab2ee admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.646929 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d453d-2e90-40f8-9ccf-24c85ad40353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.647783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d453d-2e90-40f8-9ccf-24c85ad40353 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.648003 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d453d-2e90-40f8-9ccf-24c85ad40353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.648269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d453d-2e90-40f8-9ccf-24c85ad40353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.659853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d453d-2e90-40f8-9ccf-24c85ad40353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.661341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d453d-2e90-40f8-9ccf-24c85ad40353 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.676368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1320/2630] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:01 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:01.831724 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:01.903183 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026d019-d776-4cfb-97af-06e95e124a8d None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:01.908408 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:01.909283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:01.909569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:01.909853 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] PATH_INFO: `/v3/users/231a011bff304db6bc4daa1594bec2af` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:01.911072 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=231a011bff304db6bc4daa1594bec2af)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:01.913282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:01.981936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231a011bff304db6bc4daa1594bec2af'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:01.983222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231a011bff304db6bc4daa1594bec2af'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:01.991969 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231a011bff304db6bc4daa1594bec2af'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:01.992693 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231a011bff304db6bc4daa1594bec2af'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.002054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231a011bff304db6bc4daa1594bec2af'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.011070 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:02.025821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1311/2631] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:01 2022] DELETE /identity/v3/users/231a011bff304db6bc4daa1594bec2af => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:02.030243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:02.030538 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:02.030817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:02.050135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:02.051409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:02.151158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1321/2632] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:02.165533 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:02.167864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:02.248720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df3e15-57dc-4fa9-969c-769e005bb7fa admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:02.252514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:02.253181 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:02.253487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:02.253760 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:02.265940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:02.267613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:02.278153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1312/2633] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:26:02.763338 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:02.764906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:02.785489 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f7156-1059-4d0f-9406-0e74c7a5d567 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:02.791453 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:02.792343 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:02.792629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:02.792954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] PATH_INFO: `/v3/projects/26f9cfb95ef647f5a87c22ec4048d26c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:02.794968 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f9cfb95ef647f5a87c22ec4048d26c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:02.797247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:02.857886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:02.861368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f9cfb95ef647f5a87c22ec4048d26c', 'request_id': 'req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.871932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f9cfb95ef647f5a87c22ec4048d26c', 'request_id': 'req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.881841 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f9cfb95ef647f5a87c22ec4048d26c', 'request_id': 'req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.894571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] The token cache is being invalidate because project 26f9cfb95ef647f5a87c22ec4048d26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:02.894960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.895546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:02.897112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1322/2634] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:02 2022] DELETE /identity/v3/projects/26f9cfb95ef647f5a87c22ec4048d26c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:02.901170 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:02.912547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d30d30-99b5-4ff2-aba1-ceacef6116f0 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:02.978531 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:02.979535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:02.980856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:02.981179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] PATH_INFO: `/v3/users/2be0d0e6b7824583a6f4394d1109664b` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:02.982619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be0d0e6b7824583a6f4394d1109664b)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:02.984678 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:03.042742 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be0d0e6b7824583a6f4394d1109664b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.044061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be0d0e6b7824583a6f4394d1109664b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.052551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be0d0e6b7824583a6f4394d1109664b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.053670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be0d0e6b7824583a6f4394d1109664b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.063127 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-049d9371-646e-4d96-aa2b-a9f453200c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be0d0e6b7824583a6f4394d1109664b'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.084777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1313/2635] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:02 2022] DELETE /identity/v3/users/2be0d0e6b7824583a6f4394d1109664b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:03.524486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:03.526767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:03.551001 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e88b5-5a52-4c24-95d5-675e1234cbdf admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:03.621451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:03.623049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:03.623337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:03.623616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] PATH_INFO: `/v3/projects/ed3256b8f89445c08f4b0fd4d448d4b1` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:03.624947 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed3256b8f89445c08f4b0fd4d448d4b1)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:03.627639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:03.681406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed3256b8f89445c08f4b0fd4d448d4b1', 'request_id': 'req-d381e050-bb65-497a-892e-4f6560e8842d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.692787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed3256b8f89445c08f4b0fd4d448d4b1', 'request_id': 'req-d381e050-bb65-497a-892e-4f6560e8842d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.698812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed3256b8f89445c08f4b0fd4d448d4b1', 'request_id': 'req-d381e050-bb65-497a-892e-4f6560e8842d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.711108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] The token cache is being invalidate because project ed3256b8f89445c08f4b0fd4d448d4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:03.711386 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.712033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-d381e050-bb65-497a-892e-4f6560e8842d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:03.713567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1323/2636] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:03 2022] DELETE /identity/v3/projects/ed3256b8f89445c08f4b0fd4d448d4b1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:03.718456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:03.718823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:03.719219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:03.741449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:03.742770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:03.749549 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:03.817689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1314/2637] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:03.825207 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:03.825725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:03.825828 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:03.845016 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:03.846233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:03.854426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1324/2638] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:03.861605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:03.874239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35914ad-2fc4-425a-963d-e3d47f7c872a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:03.880471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:03.881271 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:03.881659 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:03.882093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:03.882968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:03.883383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:03.885081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:03.904310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1315/2639] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:03.905541 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:03.915219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc6d1b-5609-4f41-942b-20928c641c66 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:03.920886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:03.921551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:03.921834 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:03.922107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:03.922795 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:03.925092 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:03.958494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1325/2640] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:03 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:03.959754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:03.970237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce46b5-bf86-49ed-984e-6f0f5df02feb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.044289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.045354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.045795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.046223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.047231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.051035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.075618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1316/2641] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:03 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:04.079549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.093168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecec46-c8a3-46d0-b37a-57ebc8e19125 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.099495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.100321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.100604 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.100873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.101606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.103254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.116494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1326/2642] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:04.120224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.130938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fa5df-a7a0-420b-9040-103a5976d8e3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.141631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2fa4f-aac5-4e0f-bc21-79db1ff56e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.142444 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2fa4f-aac5-4e0f-bc21-79db1ff56e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.142793 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2fa4f-aac5-4e0f-bc21-79db1ff56e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.143293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2fa4f-aac5-4e0f-bc21-79db1ff56e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.144866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2fa4f-aac5-4e0f-bc21-79db1ff56e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.146734 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2fa4f-aac5-4e0f-bc21-79db1ff56e51 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.165448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1317/2643] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:26:04.170244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.180905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3b68-6391-49ec-9d45-2009faa6fadf admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.193010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.193957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.194197 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.194475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] PATH_INFO: `/v3/projects/cab9c65461ee4f47941785fc35fdddbf/users/392795e4876e4169b9d11c59989b37ad/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.234924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab9c65461ee4f47941785fc35fdddbf, user_id=392795e4876e4169b9d11c59989b37ad, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.239358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.267188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:04.275404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1327/2644] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:04 2022] PUT /identity/v3/projects/cab9c65461ee4f47941785fc35fdddbf/users/392795e4876e4169b9d11c59989b37ad/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:04.289564 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.290633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.291739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.329348 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] MFA Rules not processed for user `392795e4876e4169b9d11c59989b37ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:04.330594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:04.487148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1318/2645] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:04.489955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.610068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a45f5-a524-477b-b42c-7866658fecc1 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.616173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.617213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.617213 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.617405 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.618052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.619974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.670892 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1328/2646] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:04 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:04.672529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.688329 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89143-cf28-40af-8818-afbb698c7667 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.693815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:04.761019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.762022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.762857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.763142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.763843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.765777 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.791695 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1319/2647] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:04 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:04.795725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.807238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07eca4-0866-4291-80fb-1d1a63f01072 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.814297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.815042 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.815324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.815595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.816326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.818036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.830624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1329/2648] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:26:04.837316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.849792 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c0f3e-4993-4cae-83e9-046ce7ca61e4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:04.855848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:04.857303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:04.857600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:04.857885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] PATH_INFO: `/v3/projects/d6277251912e481c93a4e887ddf25097/users/fc9fa1da41394098b14015bdad015ee7/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:04.876794 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6277251912e481c93a4e887ddf25097, user_id=fc9fa1da41394098b14015bdad015ee7, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:04.879754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:04.907471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1320/2649] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:04 2022] PUT /identity/v3/projects/d6277251912e481c93a4e887ddf25097/users/fc9fa1da41394098b14015bdad015ee7/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:04.911408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:04.925843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0d037-d3da-4391-b151-67410b35d598 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.004262 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f0ab4-5740-4953-82b9-4b61a49792fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.005046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f0ab4-5740-4953-82b9-4b61a49792fe admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.005401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f0ab4-5740-4953-82b9-4b61a49792fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.005756 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f0ab4-5740-4953-82b9-4b61a49792fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.006590 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f0ab4-5740-4953-82b9-4b61a49792fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.008691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f0ab4-5740-4953-82b9-4b61a49792fe admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.026701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1330/2650] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:05.031057 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.043298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d651fc-b23e-4285-804b-8f9bd4ff84f0 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.054650 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.055739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.056741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.057111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] PATH_INFO: `/v3/projects/d6277251912e481c93a4e887ddf25097/users/fc9fa1da41394098b14015bdad015ee7/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.061404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6277251912e481c93a4e887ddf25097, user_id=fc9fa1da41394098b14015bdad015ee7, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.064153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.094787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1321/2651] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:05 2022] PUT /identity/v3/projects/d6277251912e481c93a4e887ddf25097/users/fc9fa1da41394098b14015bdad015ee7/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:05.096884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6e2b-bda7-4edf-913f-8bd9e783030d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.097179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6e2b-bda7-4edf-913f-8bd9e783030d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.097469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6e2b-bda7-4edf-913f-8bd9e783030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.133005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-cbce6e2b-bda7-4edf-913f-8bd9e783030d None None] MFA Rules not processed for user `fc9fa1da41394098b14015bdad015ee7`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:05.134314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-cbce6e2b-bda7-4edf-913f-8bd9e783030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:05.236038 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:05.246765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1331/2652] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:05.258710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.260493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:05.305974 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.306231 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.306461 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.343815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] MFA Rules not processed for user `0c13da70175548fbbddc42879c3024d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:05.345413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:05.354940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ccc5d-3c35-40eb-b545-015710a21653 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.361563 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.362237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.362523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.362800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.377136 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.379182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.400860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1322/2653] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:05.437145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1332/2654] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:05.446946 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.447987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:05.528366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018db6-ed82-46e0-aaa7-d2e2b7e8d99b service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.536773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.537782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.538058 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.538338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.539158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.541952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.548514 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:05.599153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1323/2655] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:05 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:26:05.603071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.615527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c671581-98fb-422e-94b9-113b3c31e3f2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.696971 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.697975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.698629 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.699280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.700005 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.702011 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.727019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1333/2656] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:05 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:05.728405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.741296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66689dc6-3e86-4d13-8635-a34eec303245 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.746520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.747265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.747577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.747856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.748596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.750220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.762021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1324/2657] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:05.771110 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.787714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fccf9-e636-41a7-8126-a273212ae6b6 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.795236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.796235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.796579 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.796939 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] PATH_INFO: `/v3/projects/681031f6fd45473db14bcb50c37f26ad/users/546dd3d9bd24482cb5c2acf8fe4f5dcd/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.806379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.813704 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681031f6fd45473db14bcb50c37f26ad, user_id=546dd3d9bd24482cb5c2acf8fe4f5dcd, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.816449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.831927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1334/2658] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:05 2022] PUT /identity/v3/projects/681031f6fd45473db14bcb50c37f26ad/users/546dd3d9bd24482cb5c2acf8fe4f5dcd/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:05.842734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.882281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3b0d6-b99f-4ec6-a9a6-6e2128cb6e2d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.901527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:05.919600 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e58d7-c0b9-496f-943b-abcdd66fff82 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.925767 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.927015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.927325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.927607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.928347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.930958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.945765 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1335/2659] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:05.946788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.948331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc9fa1da41394098b14015bdad015ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6277251912e481c93a4e887ddf25097', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.949134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.949497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.950385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.951233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:05.951616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.954160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.957887 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce194a-0ed5-4157-922f-eb30f34b0740 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.962923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:05.963859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:05.964148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:05.964426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] PATH_INFO: `/v3/projects/681031f6fd45473db14bcb50c37f26ad/users/546dd3d9bd24482cb5c2acf8fe4f5dcd/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:05.968515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1325/2660] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:05.968652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681031f6fd45473db14bcb50c37f26ad, user_id=546dd3d9bd24482cb5c2acf8fe4f5dcd, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:05.969873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:05.970661 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:05.980094 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969d3e6-a953-45e6-840f-5345d59ec5e0 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:05.980830 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1336/2661] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:05 2022] PUT /identity/v3/projects/681031f6fd45473db14bcb50c37f26ad/users/546dd3d9bd24482cb5c2acf8fe4f5dcd/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:06.053584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc9fa1da41394098b14015bdad015ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6277251912e481c93a4e887ddf25097', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:06.054156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:06.054515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:06.054802 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:06.055558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:06.057477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:06.093993 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1326/2662] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:05 2022] POST /identity/v3/projects => generated 355 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:26:06.098206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:06.109327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c364a8-3bb4-484e-9a3c-7a690e93bfc9 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:06.191056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc9fa1da41394098b14015bdad015ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6277251912e481c93a4e887ddf25097', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:06.191967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:06.192363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:06.192778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:06.194108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:06.194108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:06.196114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:06.210877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1337/2663] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:06.263224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:06.269797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:06.276655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c592f-221c-4c86-8fcb-5d108b159922 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:06.284217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc9fa1da41394098b14015bdad015ee7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6277251912e481c93a4e887ddf25097', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:06.285326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:06.285718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:06.286193 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] PATH_INFO: `/v3/projects/8b127c8e2bb0418f971bf36d273e3dfa` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:06.287588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8b127c8e2bb0418f971bf36d273e3dfa)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:06.290774 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:06.357065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b127c8e2bb0418f971bf36d273e3dfa', 'request_id': 'req-8f24c33e-5c36-4354-a471-c4586c55190e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.370588 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b127c8e2bb0418f971bf36d273e3dfa', 'request_id': 'req-8f24c33e-5c36-4354-a471-c4586c55190e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.381680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b127c8e2bb0418f971bf36d273e3dfa', 'request_id': 'req-8f24c33e-5c36-4354-a471-c4586c55190e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.407390 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] The token cache is being invalidate because project 8b127c8e2bb0418f971bf36d273e3dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:06.407832 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.408923 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-8f24c33e-5c36-4354-a471-c4586c55190e tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.422233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1327/2664] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:06 2022] DELETE /identity/v3/projects/8b127c8e2bb0418f971bf36d273e3dfa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:06.671215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:06.673740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:06.788478 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5302700-38cb-4dd3-a647-fe77a32aa575 tempest-VolumeQuotaClassesTest-1620227604 tempest-VolumeQuotaClassesTest-1620227604-project-admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:06.863773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:06.864763 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:06.864950 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:06.865210 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] PATH_INFO: `/v3/users/392795e4876e4169b9d11c59989b37ad` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:06.867988 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=392795e4876e4169b9d11c59989b37ad)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:06.870284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:06.888598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:06.941961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '392795e4876e4169b9d11c59989b37ad'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.942804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '392795e4876e4169b9d11c59989b37ad'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.953849 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '392795e4876e4169b9d11c59989b37ad'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.954944 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '392795e4876e4169b9d11c59989b37ad'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.970321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '392795e4876e4169b9d11c59989b37ad'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:06.986883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1338/2665] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:06 2022] DELETE /identity/v3/users/392795e4876e4169b9d11c59989b37ad => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:06.989255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:06.989823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:06.989823 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:07.009605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:07.011870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:07.083067 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1328/2666] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:07.092362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:07.094492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:07.171384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff16e1-81d6-49b2-980e-94a43d6b0463 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:07.176517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:07.177325 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:07.177932 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:07.178782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:07.191900 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:07.193529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:07.205639 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1339/2667] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:26:07.581726 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:07.583297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:07.605074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa5189-c13f-4d92-9be3-4071e7d96a4b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:07.612106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:07.613174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:07.613637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:07.614071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] PATH_INFO: `/v3/projects/cab9c65461ee4f47941785fc35fdddbf` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:07.615497 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab9c65461ee4f47941785fc35fdddbf)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:07.617572 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:07.693284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab9c65461ee4f47941785fc35fdddbf', 'request_id': 'req-c20808ec-a646-4f1a-9ee8-ca56477b793c'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.700658 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab9c65461ee4f47941785fc35fdddbf', 'request_id': 'req-c20808ec-a646-4f1a-9ee8-ca56477b793c'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.710611 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab9c65461ee4f47941785fc35fdddbf', 'request_id': 'req-c20808ec-a646-4f1a-9ee8-ca56477b793c'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.725425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] The token cache is being invalidate because project cab9c65461ee4f47941785fc35fdddbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:07.726513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.726513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.731940 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1329/2668] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:07 2022] DELETE /identity/v3/projects/cab9c65461ee4f47941785fc35fdddbf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:07.733146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:07.743454 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb4eba-2b82-406b-808a-33a263e77471 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:07.812008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:07.812996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:07.813347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:07.813735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] PATH_INFO: `/v3/users/fc9fa1da41394098b14015bdad015ee7` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:07.815079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9fa1da41394098b14015bdad015ee7)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:07.817117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:07.880595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:07.889563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9fa1da41394098b14015bdad015ee7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.890165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9fa1da41394098b14015bdad015ee7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.899371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9fa1da41394098b14015bdad015ee7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.900619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9fa1da41394098b14015bdad015ee7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.914810 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-9e8989a5-d982-49d6-9d1f-2a9562041008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9fa1da41394098b14015bdad015ee7'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:07.932955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1340/2669] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:07 2022] DELETE /identity/v3/users/fc9fa1da41394098b14015bdad015ee7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:08.326388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:08.331233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:08.366855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20808ec-a646-4f1a-9ee8-ca56477b793c admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:08.376584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:08.579602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:08.581885 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:08.583178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:08.583469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] PATH_INFO: `/v3/projects/d6277251912e481c93a4e887ddf25097` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:08.586866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6277251912e481c93a4e887ddf25097)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:08.590157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:08.718474 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6277251912e481c93a4e887ddf25097', 'request_id': 'req-104b940d-ab2c-4f7b-a356-df1d615a8739'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:08.738395 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6277251912e481c93a4e887ddf25097', 'request_id': 'req-104b940d-ab2c-4f7b-a356-df1d615a8739'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:08.754041 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6277251912e481c93a4e887ddf25097', 'request_id': 'req-104b940d-ab2c-4f7b-a356-df1d615a8739'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:08.772620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] The token cache is being invalidate because project d6277251912e481c93a4e887ddf25097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:08.773276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:08.774116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-104b940d-ab2c-4f7b-a356-df1d615a8739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:08.777610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1330/2670] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:08 2022] DELETE /identity/v3/projects/d6277251912e481c93a4e887ddf25097 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:08.782981 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:08.783302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:08.783591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:08.805924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:08.807591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:08.894212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1341/2671] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:08.902431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:08.902740 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:08.903567 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:08.939347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:08.940590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:08.967279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1331/2672] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:08.971635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:08.991868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200649f-c4db-423a-9047-b234b26d7c9f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.007866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.008571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.008868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.011387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.012238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:09.015228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.019597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.053923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1342/2673] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:09.058418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.069835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c939e-f0ec-4089-82cd-e38b912bba25 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.078119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.078801 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.079656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.079934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.081159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.094925 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.167562 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1332/2674] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:09 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:09.171375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.190491 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f7b07-ba31-4ab1-956b-8f963f41804b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.349229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.349916 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.350202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.350498 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.351218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.353181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.411209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1343/2675] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:09 2022] POST /identity/v3/users => generated 291 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:09.413171 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.431586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d224ffe-1eea-4583-b6b9-13efccaa43b5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.447265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.451397 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.456029 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.456799 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.459559 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.461931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.492081 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1333/2676] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:09.496019 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.509352 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f8b01-0ee7-4e82-ba25-ce7b5ef44e92 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.516420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad8a5f-7832-42c0-81a2-a9408a07b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.517520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad8a5f-7832-42c0-81a2-a9408a07b979 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.517970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad8a5f-7832-42c0-81a2-a9408a07b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.518406 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad8a5f-7832-42c0-81a2-a9408a07b979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.519369 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad8a5f-7832-42c0-81a2-a9408a07b979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.521150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad8a5f-7832-42c0-81a2-a9408a07b979 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.541462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1344/2677] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:09.545759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.556008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cddcc-3380-4c8b-a244-420fd5df0a91 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.561539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.562441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.562729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.563035 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] PATH_INFO: `/v3/projects/4662df8567ea40f78b4d7bd225a1e309/users/ee8451adc1de4a74a3f5c2b4af72f023/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.586322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4662df8567ea40f78b4d7bd225a1e309, user_id=ee8451adc1de4a74a3f5c2b4af72f023, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.589365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.618994 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1334/2678] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:09 2022] PUT /identity/v3/projects/4662df8567ea40f78b4d7bd225a1e309/users/ee8451adc1de4a74a3f5c2b4af72f023/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:09.630156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.630474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.630880 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.666972 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] MFA Rules not processed for user `ee8451adc1de4a74a3f5c2b4af72f023`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:09.668304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:09.702662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:09.808153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1345/2679] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:09.813304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.864858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:09.903380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b81090-44ef-4e21-a6c6-412ada77632e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:09.909202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:09.910116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:09.910556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:09.911643 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:09.912706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:09.916582 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:09.965837 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1335/2680] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:09 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:09.970107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:09.982183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f075991-5f31-43bb-9663-d1abe11b483e None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:10.066888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:10.068261 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.068580 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.068965 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.069689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:10.072117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:10.110625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1346/2681] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:09 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:10.114718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:10.133288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b8b45-9345-49cc-9d00-25f39928ab92 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:10.139270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:10.140159 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.140599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.141024 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.142052 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:10.144080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:10.161442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1336/2682] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:10.165873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:10.184014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1ba2-3a2d-44b5-96f8-a085af48c3fa admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:10.190624 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:10.191691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.192039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.192381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] PATH_INFO: `/v3/projects/16a94ce9e5da42b4968dfa28d5018dce/users/d06de37283414cc199c7c60835891062/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.215317 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a94ce9e5da42b4968dfa28d5018dce, user_id=d06de37283414cc199c7c60835891062, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:10.219532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:10.244443 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1347/2683] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:10 2022] PUT /identity/v3/projects/16a94ce9e5da42b4968dfa28d5018dce/users/d06de37283414cc199c7c60835891062/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:10.248336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:10.266584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfe4bc-c430-4034-99b5-1f9e641311bb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:10.346389 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699757f2-0714-4dad-964f-1a1bee20c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:10.347211 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699757f2-0714-4dad-964f-1a1bee20c17b admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.347584 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699757f2-0714-4dad-964f-1a1bee20c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.347861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699757f2-0714-4dad-964f-1a1bee20c17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.348590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699757f2-0714-4dad-964f-1a1bee20c17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:10.350420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699757f2-0714-4dad-964f-1a1bee20c17b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:10.364635 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1337/2684] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:10.369234 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:10.383786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f3796-d456-42c1-9474-abe1104e2aaf admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:10.388362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:10.390689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.391109 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.391485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] PATH_INFO: `/v3/projects/16a94ce9e5da42b4968dfa28d5018dce/users/d06de37283414cc199c7c60835891062/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.393725 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a94ce9e5da42b4968dfa28d5018dce, user_id=d06de37283414cc199c7c60835891062, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:10.396824 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:10.409303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1348/2685] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:10 2022] PUT /identity/v3/projects/16a94ce9e5da42b4968dfa28d5018dce/users/d06de37283414cc199c7c60835891062/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:10.428126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.429235 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.429673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.473847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] MFA Rules not processed for user `d06de37283414cc199c7c60835891062`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:10.476093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:10.586657 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1338/2686] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:10.619721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:10.622273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:10.784649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39f61a-d4de-4076-9001-a65f8208afc1 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:10.800826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1ccaf-5041-488b-ad1a-7e4f12b285ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:10.801563 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1ccaf-5041-488b-ad1a-7e4f12b285ec service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:10.801788 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1ccaf-5041-488b-ad1a-7e4f12b285ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:10.802056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1ccaf-5041-488b-ad1a-7e4f12b285ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:10.829045 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1ccaf-5041-488b-ad1a-7e4f12b285ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:10.830648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1ccaf-5041-488b-ad1a-7e4f12b285ec service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:10.857278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1349/2687] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:11.594399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:11.645638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:11.686260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c7dc5-0804-46bf-b574-dcc2b7b9a263 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:11.692257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:11.693347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:11.693743 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:11.694325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] PATH_INFO: `/v3/users/ee8451adc1de4a74a3f5c2b4af72f023` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:11.696000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee8451adc1de4a74a3f5c2b4af72f023)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:11.699327 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:11.781157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee8451adc1de4a74a3f5c2b4af72f023'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:11.782110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee8451adc1de4a74a3f5c2b4af72f023'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:11.793370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee8451adc1de4a74a3f5c2b4af72f023'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:11.794859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee8451adc1de4a74a3f5c2b4af72f023'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:11.807746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee8451adc1de4a74a3f5c2b4af72f023'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:11.834934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1339/2688] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:11 2022] DELETE /identity/v3/users/ee8451adc1de4a74a3f5c2b4af72f023 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:11.837783 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9267ef9-53cb-4627-a221-561064ebbd12 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:11.838270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9267ef9-53cb-4627-a221-561064ebbd12 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:11.838401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9267ef9-53cb-4627-a221-561064ebbd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:11.853925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:11.863027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f9267ef9-53cb-4627-a221-561064ebbd12 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:11.865140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f9267ef9-53cb-4627-a221-561064ebbd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:11.982129 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1350/2689] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:11.989223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:11.990861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:12.143172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879da62-630e-4df7-8f84-0ebfaa3115b0 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:12.148113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:12.149112 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:12.149499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:12.149872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:12.164279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:12.166038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:12.180494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1340/2690] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:12.301277 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:12.301557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:12.301820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:12.344595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] MFA Rules not processed for user `546dd3d9bd24482cb5c2acf8fe4f5dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:12.347297 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:12.503599 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1351/2691] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:12.524246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:12.525172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:12.629939 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a0f87-b95d-4478-a91c-b9806616b451 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:12.633838 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fae9cab0b834583a01d0b57c78257af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:12.634498 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:12.634798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:12.635086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:12.637066 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:12.645234 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:12.684791 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:12.686154 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:12.706287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189ce4a-dbb5-4029-8ff8-1ab0bc3de4b1 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:12.727230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:12.730771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:12.732449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:12.732637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:12.732853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:12.733958 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] PATH_INFO: `/v3/projects/4662df8567ea40f78b4d7bd225a1e309` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:12.735523 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4662df8567ea40f78b4d7bd225a1e309)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:12.737891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:12.750191 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1341/2692] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:12.812519 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4662df8567ea40f78b4d7bd225a1e309', 'request_id': 'req-263eece4-8534-498e-9ce4-585d34dc655c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:12.827100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4662df8567ea40f78b4d7bd225a1e309', 'request_id': 'req-263eece4-8534-498e-9ce4-585d34dc655c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:12.847465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4662df8567ea40f78b4d7bd225a1e309', 'request_id': 'req-263eece4-8534-498e-9ce4-585d34dc655c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:12.876951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] The token cache is being invalidate because project 4662df8567ea40f78b4d7bd225a1e309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:12.877250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:12.878140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-263eece4-8534-498e-9ce4-585d34dc655c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:12.882711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1352/2693] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:12 2022] DELETE /identity/v3/projects/4662df8567ea40f78b4d7bd225a1e309 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:12.885198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:12.900508 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b402230-7e08-475a-a8b8-e298219f9652 service glance] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:12.907462 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:12.907816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:12.908180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:12.930253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:12.937422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:12.949111 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:12.979133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:12.980119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:12.980479 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:12.980818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] PATH_INFO: `/v3/users/d06de37283414cc199c7c60835891062` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:12.982167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06de37283414cc199c7c60835891062)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:12.984805 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:13.029645 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1353/2694] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:26:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:13.077284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06de37283414cc199c7c60835891062'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.077958 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06de37283414cc199c7c60835891062'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.082182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:13.083893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:13.092103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06de37283414cc199c7c60835891062'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.092840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06de37283414cc199c7c60835891062'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.113755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06de37283414cc199c7c60835891062'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.153727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1342/2695] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:12 2022] DELETE /identity/v3/users/d06de37283414cc199c7c60835891062 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:13.188014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01e206-85f2-48ed-b252-26555cb9b1f2 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:13.268455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:13.269270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:13.269615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:13.269954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:13.272770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:13.356957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:13.358565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060eb267-5950-4047-9a8b-81893bc542dc service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:13.368298 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1354/2696] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:26:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 07 13:26:13.458866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:13.460415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:13.522240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:13.580095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022385fa-fc58-4f2d-9e73-b76ec71d8400 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:13.587496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:13.588380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:13.588672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:13.588989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] PATH_INFO: `/v3/projects/16a94ce9e5da42b4968dfa28d5018dce` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:13.590322 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a94ce9e5da42b4968dfa28d5018dce)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:13.592610 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:13.654030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a94ce9e5da42b4968dfa28d5018dce', 'request_id': 'req-74bdc887-7950-4dcd-9c91-429f4918b987'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.664388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a94ce9e5da42b4968dfa28d5018dce', 'request_id': 'req-74bdc887-7950-4dcd-9c91-429f4918b987'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.679722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a94ce9e5da42b4968dfa28d5018dce', 'request_id': 'req-74bdc887-7950-4dcd-9c91-429f4918b987'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.703106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] The token cache is being invalidate because project 16a94ce9e5da42b4968dfa28d5018dce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:13.703646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.704466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-74bdc887-7950-4dcd-9c91-429f4918b987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:13.706590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1343/2697] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:13 2022] DELETE /identity/v3/projects/16a94ce9e5da42b4968dfa28d5018dce => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:13.711640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:13.712183 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:13.712640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:13.725951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:13.758720 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:13.761034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:13.862619 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1355/2698] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:13.869856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:13.870150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:13.870447 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:13.895603 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:13.896822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:13.908718 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1344/2699] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:13.912837 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:13.926672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a41d6-905a-4932-873d-59170ecc6665 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:13.944187 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:13.944883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:13.945168 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:13.945440 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:13.946185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:13.946493 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:13.948179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:13.966052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1356/2700] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:13.967137 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:13.977333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5deb54e-8c3c-447c-92eb-17be5bc7bf62 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:13.985155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:13.985990 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:13.986536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:13.986815 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:13.988652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:13.989653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:14.026336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1345/2701] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:13 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:14.031570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:14.045222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac3305-00b0-41ea-b5bd-26ba2987df4d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:14.143513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:14.144101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.144451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.144804 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.146046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:14.157156 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:14.194976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1357/2702] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:14 2022] POST /identity/v3/users => generated 281 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:14.199316 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:14.225107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600471eb-2183-4801-92ec-969073c30a9f admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:14.252431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:14.253312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.254180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.254561 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.257297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:14.259184 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:14.276717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1346/2703] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:26:14.278691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:14.293940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1e597-0c73-4275-b7e3-71877093d576 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:14.300230 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49664cb-e54f-4c69-8e08-e546d58c4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:14.301248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49664cb-e54f-4c69-8e08-e546d58c4501 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.302451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49664cb-e54f-4c69-8e08-e546d58c4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.303272 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49664cb-e54f-4c69-8e08-e546d58c4501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.304853 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49664cb-e54f-4c69-8e08-e546d58c4501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:14.307675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49664cb-e54f-4c69-8e08-e546d58c4501 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:14.329484 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1358/2704] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:26:14.330546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:14.346755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e151c-eca3-4117-84fe-6cfd0078dd0e admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:14.357141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:14.358053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.358340 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.358621 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] PATH_INFO: `/v3/projects/e67f36e7f2b34e34bb3738d8ebd4ef4b/users/f62b1b07cced4f718d484cd49a140937/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.384764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67f36e7f2b34e34bb3738d8ebd4ef4b, user_id=f62b1b07cced4f718d484cd49a140937, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:14.388461 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:14.421219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1347/2705] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:14 2022] PUT /identity/v3/projects/e67f36e7f2b34e34bb3738d8ebd4ef4b/users/f62b1b07cced4f718d484cd49a140937/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:14.498311 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.499366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.499366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.537034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] MFA Rules not processed for user `f62b1b07cced4f718d484cd49a140937`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:14.538279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:14.657161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:14.665002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1359/2706] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:14.701863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:14.705088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:14.808655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c0597-3e9e-4735-9359-e81418d195d7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:14.813717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d92ca5-a187-497a-a00b-f3721ede05c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:14.814664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d92ca5-a187-497a-a00b-f3721ede05c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.815197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d92ca5-a187-497a-a00b-f3721ede05c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.815708 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d92ca5-a187-497a-a00b-f3721ede05c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.827358 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:14.829499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d92ca5-a187-497a-a00b-f3721ede05c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:14.831465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d92ca5-a187-497a-a00b-f3721ede05c1 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:14.851011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1348/2707] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:14.868375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:14.958274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5b6ab-0521-4bb6-bee2-85fadeb62fcd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:14.964036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:14.965162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:14.972940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:14.973303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] PATH_INFO: `/v3/users/f62b1b07cced4f718d484cd49a140937` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:14.975368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f62b1b07cced4f718d484cd49a140937)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:14.977652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:15.047280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f62b1b07cced4f718d484cd49a140937'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:15.049702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f62b1b07cced4f718d484cd49a140937'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:15.061480 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f62b1b07cced4f718d484cd49a140937'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:15.062209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f62b1b07cced4f718d484cd49a140937'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:15.076748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f62b1b07cced4f718d484cd49a140937'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:15.102278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1360/2708] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:26:14 2022] DELETE /identity/v3/users/f62b1b07cced4f718d484cd49a140937 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:15.108416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:15.108840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:15.110451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:15.143977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:15.145315 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:15.252206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1349/2709] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:15.261657 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:15.263331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:15.348863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06d0f0-a543-4733-8014-285b8720212b admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:15.355994 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:15.358371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:15.358906 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:15.360481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:15.375637 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:15.377371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:15.391700 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1361/2710] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:15.932440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:15.942741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:15.945400 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:15.962218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb349fe-db1b-4cea-9f62-677324072591 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:15.967486 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:15.967997 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae19f11-3a19-419e-9648-bf1e07befd84 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:15.968587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:15.968881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:15.969162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:15.971457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:15.978785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:15.980031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:15.980031 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:15.980427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] PATH_INFO: `/v3/projects/e67f36e7f2b34e34bb3738d8ebd4ef4b` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:15.982079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67f36e7f2b34e34bb3738d8ebd4ef4b)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:15.984551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.005987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:16.069036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67f36e7f2b34e34bb3738d8ebd4ef4b', 'request_id': 'req-3887c1c9-08bf-4434-a911-7ea76d3a533d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:16.088102 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67f36e7f2b34e34bb3738d8ebd4ef4b', 'request_id': 'req-3887c1c9-08bf-4434-a911-7ea76d3a533d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:16.096940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67f36e7f2b34e34bb3738d8ebd4ef4b', 'request_id': 'req-3887c1c9-08bf-4434-a911-7ea76d3a533d'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:16.107490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.109000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05aee6-10be-4f51-97b3-b1b6f0d3edc9 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.130144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1350/2711] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:16.153996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] The token cache is being invalidate because project e67f36e7f2b34e34bb3738d8ebd4ef4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:26:16.154391 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:16.155323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-3887c1c9-08bf-4434-a911-7ea76d3a533d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:26:16.170564 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1362/2712] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:26:15 2022] DELETE /identity/v3/projects/e67f36e7f2b34e34bb3738d8ebd4ef4b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:26:16.172480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.172887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.173298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.212671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:16.214284 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:16.326655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1351/2713] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:16.328803 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.329242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.329726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.364050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:16.365452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:16.389695 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1363/2714] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:16.391418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:16.405585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c29865-d530-425a-9350-b9229fbdba25 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:16.423098 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:16.423788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.424071 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.424347 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.425095 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:26:16.425401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.427080 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.490883 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1352/2715] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:26:16.491107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:16.503752 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631e482-5b9a-4480-b1e4-f41fb853b895 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:16.509452 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:16.510370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.510861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.511301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.512503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.514922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.557497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1364/2716] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:26:16 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:16.558101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:16.571476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059161a9-f74c-4b79-86e2-a9c076662bc0 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:16.596190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:16.689819 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:16.690627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.691021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.691365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.692130 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.694763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.732305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:16.744210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1353/2717] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:26:16 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:26:16.753077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3c300-301c-4cb9-bf2d-9bbc86ccab93 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:16.758492 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:16.759202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.759486 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.759764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.760495 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.762190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.780912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:16.793439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f35a80-ce07-4732-a1c4-eead2b4a8a41 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:16.794255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1365/2718] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:26:16.804714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9d745-8098-4100-afb4-54f54992ec65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:16.805539 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d745-8098-4100-afb4-54f54992ec65 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.805818 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d745-8098-4100-afb4-54f54992ec65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.806093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d745-8098-4100-afb4-54f54992ec65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.807030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9d745-8098-4100-afb4-54f54992ec65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.808665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9d745-8098-4100-afb4-54f54992ec65 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.824452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1354/2719] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:26:16.828528 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:16.839620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b808fb2-11f7-4b67-8e74-e010d11f69fe admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:16.846249 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:16.847531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.847531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.847792 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] PATH_INFO: `/v3/projects/a004938108be44fd8367b8fae5592613/users/625ae02165844f9baa4d6672eafbb6d6/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.866136 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a004938108be44fd8367b8fae5592613, user_id=625ae02165844f9baa4d6672eafbb6d6, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:16.870330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:16.900736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1366/2720] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:26:16 2022] PUT /identity/v3/projects/a004938108be44fd8367b8fae5592613/users/625ae02165844f9baa4d6672eafbb6d6/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:26:16.902419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ca74a-b72c-4776-8897-c878ad7898e1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:16.902715 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ca74a-b72c-4776-8897-c878ad7898e1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:16.903665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ca74a-b72c-4776-8897-c878ad7898e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:16.926543 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e93ca74a-b72c-4776-8897-c878ad7898e1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:16.928203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e93ca74a-b72c-4776-8897-c878ad7898e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:17.004591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1355/2721] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:17.013653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:17.015736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:17.088346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc9ff-df6a-49cd-be4c-b1a1823ef05f admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:17.093229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:17.093914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:17.094195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:17.094469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:17.107532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:17.109040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:17.126266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1367/2722] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:18.379473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6807d2-6937-4866-a3b8-6f6ca6e34195 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:18.380456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6807d2-6937-4866-a3b8-6f6ca6e34195 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:18.380831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6807d2-6937-4866-a3b8-6f6ca6e34195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:18.414997 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4c6807d2-6937-4866-a3b8-6f6ca6e34195 None None] MFA Rules not processed for user `625ae02165844f9baa4d6672eafbb6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:18.416318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4c6807d2-6937-4866-a3b8-6f6ca6e34195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:18.512350 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1356/2723] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:18.520444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:18.523593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:18.666805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:18.691413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c171a-d571-42d4-9b58-2791e12c32ab service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:18.705387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81760401-15c7-4dc3-afca-f781200684a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:18.706418 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81760401-15c7-4dc3-afca-f781200684a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:18.706908 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81760401-15c7-4dc3-afca-f781200684a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:18.707363 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81760401-15c7-4dc3-afca-f781200684a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:18.717740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81760401-15c7-4dc3-afca-f781200684a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:18.719424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81760401-15c7-4dc3-afca-f781200684a2 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:18.731503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1368/2724] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:26:20.268317 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf0be-aa0a-44d4-9d83-79cc23926443 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:20.268899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf0be-aa0a-44d4-9d83-79cc23926443 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:20.269250 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf0be-aa0a-44d4-9d83-79cc23926443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:20.290882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-acfaf0be-aa0a-44d4-9d83-79cc23926443 None None] MFA Rules not processed for user `625ae02165844f9baa4d6672eafbb6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:20.292110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-acfaf0be-aa0a-44d4-9d83-79cc23926443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:20.302930 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1357/2725] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:26:24.533318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:24.533812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:24.533936 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:24.551549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] MFA Rules not processed for user `0c13da70175548fbbddc42879c3024d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:24.552755 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:24.609056 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1369/2726] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:26:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:27.564382 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:27.564892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:27.564999 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:27.583279 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:26:27.584721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:27.598746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:27.642773 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1358/2727] 10.222.0.61 () {58 vars in 1009 bytes} [Tue Jun 7 13:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:26:27.655893 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:27.657379 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:27.727424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170dfca-3685-4ef4-836c-68fefafc6139 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:27.733347 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:27.733626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:27.733819 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:27.734099 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:27.744657 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:27.746269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:27.755914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1370/2728] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:26:37.635696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=16 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:37.752062 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=63 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:26:57.126800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:26:57.128492 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:57.139842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388c213-ab35-4b6a-b463-789088af4f69 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:26:57.144053 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:26:57.144685 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:26:57.144977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:26:57.145254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:26:57.146992 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:26:57.208359 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:26:57.209781 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861f014-30dc-4bd3-9ed1-35b5c86e17b6 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:26:57.219228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1359/2729] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:26:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:27:07.215030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=30 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:27:12.043854 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:27:12.045244 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:27:12.111925 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f74b82-38c6-4fbf-982d-16d7b0507657 service swift] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:27:12.117206 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:27:12.117843 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:27:12.118127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:27:12.118407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:27:12.120185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:27:12.131977 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:27:12.133542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9391d91-f03e-4aa1-a2de-84a9c02d7872 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:27:12.143896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1371/2730] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:27:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:27:22.139125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=32 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:27:38.128313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:27:38.128814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:27:38.128928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:27:38.146119 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] MFA Rules not processed for user `625ae02165844f9baa4d6672eafbb6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:27:38.147334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:27:38.154107 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1360/2731] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:27:48.141293 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:27:53.385399 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:27:53.385919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:27:53.386029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:27:53.403479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] MFA Rules not processed for user `a5766b86063f4955a642ae59e28ee226`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:27:53.404693 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:27:53.459328 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1372/2732] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:27:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:27:59.655668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:27:59.666520 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1256692-567b-4284-bc17-983d0fdca35a None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:27:59.670763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:27:59.671660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:27:59.671952 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:27:59.672228 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:27:59.673995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:27:59.748265 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:27:59.749714 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:27:59.760936 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1361/2733] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:27:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:28:02.105303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:02.115358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfdb4c4-129f-4f75-9b31-f967a960f1b6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:02.118953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:02.119587 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:02.119868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:02.120141 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:02.121920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:02.136812 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:02.138232 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:02.150877 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1373/2734] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:28:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:28:09.755892 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=34 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:12.146028 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=31 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:19.243709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:19.252434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0c177-a1e7-42b8-8526-91414953a079 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:19.256353 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:19.257457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:19.266547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:19.266547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:19.266547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:19.275326 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:19.276776 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:19.287449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1362/2735] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:28:22.079634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:22.081481 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:22.104995 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1c696-e142-4dfa-9c9f-6cae76b15ce7 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:22.126125 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:22.127299 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:22.128182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:22.128706 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:22.131120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:22.158955 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:22.159970 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:22.174333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1374/2736] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:28:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:28:25.510654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:25.512735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:25.528631 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd204a7-333e-4952-81a9-aaf14533423f service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:25.533456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:25.534380 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:25.534667 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:25.534980 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] PATH_INFO: `/v3/users/36c5458a18f146be87207e0f6dd598eb` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:25.536197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36c5458a18f146be87207e0f6dd598eb)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:25.538182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:25.584555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36c5458a18f146be87207e0f6dd598eb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.585209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36c5458a18f146be87207e0f6dd598eb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.592215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36c5458a18f146be87207e0f6dd598eb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.592845 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36c5458a18f146be87207e0f6dd598eb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.600863 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36c5458a18f146be87207e0f6dd598eb'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.617995 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1363/2737] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:28:25 2022] DELETE /identity/v3/users/36c5458a18f146be87207e0f6dd598eb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:28:25.791076 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:25.802395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df413c52-8fb4-43ba-90bd-e83f39ec9832 service placement] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:25.859724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:25.860591 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:25.860874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:25.861153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] PATH_INFO: `/v3/projects/b562a09561af4197bf2835b31a9dd34e` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:25.862333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b562a09561af4197bf2835b31a9dd34e)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:25.864351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:25.909465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b562a09561af4197bf2835b31a9dd34e', 'request_id': 'req-46fbd31a-965f-4725-a7ee-e3faf3e327fb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.916490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b562a09561af4197bf2835b31a9dd34e', 'request_id': 'req-46fbd31a-965f-4725-a7ee-e3faf3e327fb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.922455 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b562a09561af4197bf2835b31a9dd34e', 'request_id': 'req-46fbd31a-965f-4725-a7ee-e3faf3e327fb'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.933201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] The token cache is being invalidate because project b562a09561af4197bf2835b31a9dd34e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:28:25.933488 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.934818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:25.936631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1375/2738] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:28:25 2022] DELETE /identity/v3/projects/b562a09561af4197bf2835b31a9dd34e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:28:25.940648 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:25.942078 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:25.955534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdeaca7-784e-4f6a-a907-9d101ab2c161 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:26.011739 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:26.012653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:26.012942 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:26.013223 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] PATH_INFO: `/v3/users/3ed5603ee21d4999a3a94304b84dbff1` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:26.014441 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed5603ee21d4999a3a94304b84dbff1)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:26.016399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:26.061510 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed5603ee21d4999a3a94304b84dbff1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.062087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed5603ee21d4999a3a94304b84dbff1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.068998 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed5603ee21d4999a3a94304b84dbff1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.069627 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed5603ee21d4999a3a94304b84dbff1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.073500 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:26.077706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed5603ee21d4999a3a94304b84dbff1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.097943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1364/2739] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:28:25 2022] DELETE /identity/v3/users/3ed5603ee21d4999a3a94304b84dbff1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:28:26.292310 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:26.294350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:26.309608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbd31a-965f-4725-a7ee-e3faf3e327fb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:26.367095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:26.367978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:26.368283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:26.368561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] PATH_INFO: `/v3/projects/08b3a41c875d4efa9e5dade725f4e828` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:26.369735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b3a41c875d4efa9e5dade725f4e828)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:26.371692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:26.392536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:26.418987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b3a41c875d4efa9e5dade725f4e828', 'request_id': 'req-e72fb68e-943b-41d5-873e-35cebfb36e38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.426357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b3a41c875d4efa9e5dade725f4e828', 'request_id': 'req-e72fb68e-943b-41d5-873e-35cebfb36e38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.434366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b3a41c875d4efa9e5dade725f4e828', 'request_id': 'req-e72fb68e-943b-41d5-873e-35cebfb36e38'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.445375 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] The token cache is being invalidate because project 08b3a41c875d4efa9e5dade725f4e828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:28:26.445659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.446290 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:26.447838 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1376/2740] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:28:26 2022] DELETE /identity/v3/projects/08b3a41c875d4efa9e5dade725f4e828 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:28:36.094535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:36.442356 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:37.053600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:37.055109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:37.068370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a95889-0222-4f0e-8b62-8c0df7ff4ed6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:37.126177 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:37.127086 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:37.127373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:37.127653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] PATH_INFO: `/v3/users/625ae02165844f9baa4d6672eafbb6d6` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:37.128840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=625ae02165844f9baa4d6672eafbb6d6)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:37.130759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:37.182276 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625ae02165844f9baa4d6672eafbb6d6'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.182971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625ae02165844f9baa4d6672eafbb6d6'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.191602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625ae02165844f9baa4d6672eafbb6d6'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.192394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625ae02165844f9baa4d6672eafbb6d6'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.202800 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625ae02165844f9baa4d6672eafbb6d6'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.222144 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1365/2741] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:28:37 2022] DELETE /identity/v3/users/625ae02165844f9baa4d6672eafbb6d6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:28:37.409196 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:37.410635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:37.427222 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb68e-943b-41d5-873e-35cebfb36e38 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:37.486748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:37.487642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:37.487923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:37.488198 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] PATH_INFO: `/v3/projects/a004938108be44fd8367b8fae5592613` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:37.489424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a004938108be44fd8367b8fae5592613)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:37.491387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:37.542856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a004938108be44fd8367b8fae5592613', 'request_id': 'req-c7899550-ad82-40c0-b1ec-e269bab13f74'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.550821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a004938108be44fd8367b8fae5592613', 'request_id': 'req-c7899550-ad82-40c0-b1ec-e269bab13f74'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.556592 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a004938108be44fd8367b8fae5592613', 'request_id': 'req-c7899550-ad82-40c0-b1ec-e269bab13f74'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.568717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] The token cache is being invalidate because project a004938108be44fd8367b8fae5592613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:28:37.569021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.569707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:28:37.571333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1377/2742] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:28:37 2022] DELETE /identity/v3/projects/a004938108be44fd8367b8fae5592613 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:28:47.218415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:47.565373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:28:53.143874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:53.145420 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:53.220426 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603ea59-3651-402e-8680-3dbfcce4c391 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:53.290763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:53.291456 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:53.291787 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:53.292074 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:53.294217 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:53.309361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:53.310899 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:53.333654 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1366/2743] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:28:53.752740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:53.754169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:53.841155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7899550-ad82-40c0-b1ec-e269bab13f74 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:53.846596 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:53.847315 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:53.847593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:53.847884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:53.849864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:53.913702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:53.915344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4d7cf-3525-44fe-a6b7-3038bf1ba100 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:53.925714 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1378/2744] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:28:59.572335 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:28:59.575697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:59.596089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f655a-ed1d-48e8-a08d-85aec1705f7d service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:28:59.602129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:28:59.602784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:28:59.603101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:28:59.603383 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:28:59.606452 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:28:59.620449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:28:59.622069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:28:59.633298 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1367/2745] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:28:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:00.479186 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:00.479703 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:00.479809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:00.496876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] MFA Rules not processed for user `7af822c04e9747888820eebdb19ffea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:00.498075 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:00.553617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1379/2746] 10.222.0.61 () {58 vars in 967 bytes} [Tue Jun 7 13:29:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:00.593691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:00.595142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:00.665343 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79858749-808c-4c08-ae9a-1add129bfc7f service nova] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:00.670351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67519cd46f644a03a35cf1151287752e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:00.671022 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:00.671311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:00.671589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:00.679701 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:00.681291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:00.689856 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1368/2747] 10.222.0.61 () {60 vars in 1384 bytes} [Tue Jun 7 13:29:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 07 13:29:04.146652 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:04.148711 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:04.221292 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=100 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:04.263883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be75da-7fc2-4c9c-b190-3d338206dec6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:04.269362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:04.270036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:04.270458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:04.270743 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:04.272605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:04.288676 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:04.290306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:04.305218 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1380/2748] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:07.458385 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:07.459972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:07.500780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:07.527296 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e4b6a-799d-49bd-9198-b888547500a7 service swift] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:07.532305 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:07.533196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:07.533481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:07.533755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] PATH_INFO: `/v3/users/0c13da70175548fbbddc42879c3024d5` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:07.534987 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c13da70175548fbbddc42879c3024d5)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:07.536918 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:07.585440 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c13da70175548fbbddc42879c3024d5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.586424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c13da70175548fbbddc42879c3024d5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.602637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c13da70175548fbbddc42879c3024d5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.603547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c13da70175548fbbddc42879c3024d5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.612499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c13da70175548fbbddc42879c3024d5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.632297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1369/2749] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:07 2022] DELETE /identity/v3/users/0c13da70175548fbbddc42879c3024d5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:07.810608 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:07.822567 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5364-07a8-436d-8e6b-4d27510abe37 service placement] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:07.881069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:07.881935 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:07.882220 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:07.882496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] PATH_INFO: `/v3/projects/6f612ecb68c2414f92e3887ffeff0cdd` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:07.883724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f612ecb68c2414f92e3887ffeff0cdd)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:07.885644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:07.935007 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f612ecb68c2414f92e3887ffeff0cdd', 'request_id': 'req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.943048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f612ecb68c2414f92e3887ffeff0cdd', 'request_id': 'req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.950270 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f612ecb68c2414f92e3887ffeff0cdd', 'request_id': 'req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.962546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] The token cache is being invalidate because project 6f612ecb68c2414f92e3887ffeff0cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:07.962546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.962960 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:07.964344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1381/2750] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:07 2022] DELETE /identity/v3/projects/6f612ecb68c2414f92e3887ffeff0cdd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:07.968248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:07.969656 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:07.984142 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8bfd5-42f3-46e8-a0be-e9cf93d29e35 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:08.045722 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:08.046707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:08.046878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:08.047260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] PATH_INFO: `/v3/users/546dd3d9bd24482cb5c2acf8fe4f5dcd` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:08.048494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546dd3d9bd24482cb5c2acf8fe4f5dcd)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:08.050425 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:08.111248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546dd3d9bd24482cb5c2acf8fe4f5dcd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.111735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546dd3d9bd24482cb5c2acf8fe4f5dcd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.121448 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546dd3d9bd24482cb5c2acf8fe4f5dcd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.122216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546dd3d9bd24482cb5c2acf8fe4f5dcd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.124281 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:08.147393 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546dd3d9bd24482cb5c2acf8fe4f5dcd'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.170417 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1370/2751] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:07 2022] DELETE /identity/v3/users/546dd3d9bd24482cb5c2acf8fe4f5dcd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:08.362821 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:08.364380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:08.378447 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d70d87-94ed-4c35-b1c2-edfe7bd36766 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:08.439264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:08.440171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:08.440468 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:08.440760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] PATH_INFO: `/v3/projects/681031f6fd45473db14bcb50c37f26ad` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:08.441982 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=681031f6fd45473db14bcb50c37f26ad)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:08.443980 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:08.446426 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:08.493483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681031f6fd45473db14bcb50c37f26ad', 'request_id': 'req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.501582 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681031f6fd45473db14bcb50c37f26ad', 'request_id': 'req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.507046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681031f6fd45473db14bcb50c37f26ad', 'request_id': 'req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.517953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] The token cache is being invalidate because project 681031f6fd45473db14bcb50c37f26ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:08.518265 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.518921 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:08.520518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1382/2752] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:08 2022] DELETE /identity/v3/projects/681031f6fd45473db14bcb50c37f26ad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:18.165754 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=8 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:18.514987 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:24.598196 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:24.599646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:24.658612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec4951-c7ee-4e55-9430-320597101be7 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:24.706758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:24.707505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:24.707791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:24.708061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:24.709873 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:24.771047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:24.772477 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:24.781874 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1371/2753] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:28.142458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:28.144179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:28.159331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0c63-d653-42f5-9324-bd7e132e1b1a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:28.165469 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:28.166975 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.167459 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.167754 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] PATH_INFO: `/v3/users/a5766b86063f4955a642ae59e28ee226` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.169086 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5766b86063f4955a642ae59e28ee226)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:28.171148 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:28.229407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5766b86063f4955a642ae59e28ee226'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.230146 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5766b86063f4955a642ae59e28ee226'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.243344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5766b86063f4955a642ae59e28ee226'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.244128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5766b86063f4955a642ae59e28ee226'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.257677 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ed880e7b-03b1-407c-b715-8cb4f28520cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5766b86063f4955a642ae59e28ee226'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.273650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1383/2754] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:28 2022] DELETE /identity/v3/users/a5766b86063f4955a642ae59e28ee226 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:28.464896 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:28.475844 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7127c-611c-4cab-b208-4261542324c9 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:28.547738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:28.549275 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.549744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.550164 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] PATH_INFO: `/v3/projects/5541835571f64b4dbec6cabbbd6b13ca` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.552062 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5541835571f64b4dbec6cabbbd6b13ca)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:28.555273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:28.568733 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=100 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:28.620842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5541835571f64b4dbec6cabbbd6b13ca', 'request_id': 'req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.627605 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5541835571f64b4dbec6cabbbd6b13ca', 'request_id': 'req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.635975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5541835571f64b4dbec6cabbbd6b13ca', 'request_id': 'req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.647578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] The token cache is being invalidate because project 5541835571f64b4dbec6cabbbd6b13ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:28.647901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.648560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a6b5a16b-77bf-451d-b679-1ffb2e5c83db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:28.650634 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1372/2755] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:28 2022] DELETE /identity/v3/projects/5541835571f64b4dbec6cabbbd6b13ca => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:28.655588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.655876 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.656149 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.674683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:28.675930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:28.740499 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1384/2756] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:28.750124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.750388 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.750671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.770123 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:28.771399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:28.781294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1373/2757] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:28.785716 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:28.795954 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4f748-2e66-4681-90e2-758eafd44f31 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:28.800659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:28.801318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.801598 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.801868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.802595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:29:28.802928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:28.804520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:28.816431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1385/2758] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:29:28.821482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:28.830935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86201c-0c7c-44e5-ae21-41a67c6dc88b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:28.838928 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:28.839746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.840038 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.840338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.841088 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:28.843118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:28.875780 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1374/2759] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:29:28 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:29:28.880425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:28.891793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6860e1c-c8dd-43cd-b7fd-ce5f1f0097e4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:28.949660 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:28.950331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:28.950612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:28.950915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:28.951671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:28.953584 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:28.960422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:28.976474 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1386/2760] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:29:28 2022] POST /identity/v3/users => generated 278 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:28.981392 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:28.993019 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ebff9-9e30-4da4-acc7-b67b5e129cce admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:28.998981 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:28.999887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.000173 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.000599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.001372 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.002989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.018049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1375/2761] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:29.022741 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.034202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb0fd-2e96-4afc-b735-35ff62922abb admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.040201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4728b71-4a68-4239-81da-3d2c67f75137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.040874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4728b71-4a68-4239-81da-3d2c67f75137 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.041159 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4728b71-4a68-4239-81da-3d2c67f75137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.041433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4728b71-4a68-4239-81da-3d2c67f75137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.042163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4728b71-4a68-4239-81da-3d2c67f75137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.043818 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4728b71-4a68-4239-81da-3d2c67f75137 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.054745 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1387/2762] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:29.059034 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.069351 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7a8d1-1b29-43cc-b341-6d8d8748ee3b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.074128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.075061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.075346 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.075625 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] PATH_INFO: `/v3/projects/47bab14623e646acb4f676dae1107ead/users/f53f617dcc5d4b1a966989ef3dea5075/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.092727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bab14623e646acb4f676dae1107ead, user_id=f53f617dcc5d4b1a966989ef3dea5075, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.095579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.110012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1376/2763] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:29 2022] PUT /identity/v3/projects/47bab14623e646acb4f676dae1107ead/users/f53f617dcc5d4b1a966989ef3dea5075/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:29:29.120190 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.120500 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.120747 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.147551 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] MFA Rules not processed for user `f53f617dcc5d4b1a966989ef3dea5075`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:29.149734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:29.214903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1388/2764] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:29.219401 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.285482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61db68-55f0-4f22-9a15-2e58e46a0110 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.292215 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.293321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.293764 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.294283 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.295699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.299116 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.327602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1377/2765] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:29:29 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:29.332027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.344420 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e033278-e033-41f1-a797-5efbc0a08d18 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.403860 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.404446 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.404734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.405012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.405708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.407736 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.436367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1389/2766] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:29:29 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:29:29.439699 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.450763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9331ef-2ee8-4072-8dd7-f23a3266f755 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.455880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.456568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.456857 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.457131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.457910 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.459538 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.472443 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1378/2767] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:29.476855 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.488169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138e78f-7b0a-43f4-a76a-88cc43e2be76 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.493107 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.494293 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.494585 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.494883 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] PATH_INFO: `/v3/projects/2fb0eb9a63b149f49cfd078eff2f17bf/users/f65910b1bd7b4a1f8dd064a65c91f102/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.512840 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb0eb9a63b149f49cfd078eff2f17bf, user_id=f65910b1bd7b4a1f8dd064a65c91f102, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.515568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.529927 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1390/2768] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:29 2022] PUT /identity/v3/projects/2fb0eb9a63b149f49cfd078eff2f17bf/users/f65910b1bd7b4a1f8dd064a65c91f102/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:29.534154 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.544630 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7a17-a77c-4f2b-9c99-42ae1ec3df58 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.548552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:29.622277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66faaa1-304c-4446-902e-2867672b8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.623023 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66faaa1-304c-4446-902e-2867672b8840 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.623311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66faaa1-304c-4446-902e-2867672b8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.623593 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66faaa1-304c-4446-902e-2867672b8840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.624336 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66faaa1-304c-4446-902e-2867672b8840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.626003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66faaa1-304c-4446-902e-2867672b8840 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.637829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1379/2769] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:29.642522 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.655274 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e8cb2-4f12-406a-a0b9-1993d992bc4e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.660073 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.660969 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.661250 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.661521 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] PATH_INFO: `/v3/projects/2fb0eb9a63b149f49cfd078eff2f17bf/users/f65910b1bd7b4a1f8dd064a65c91f102/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.663527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb0eb9a63b149f49cfd078eff2f17bf, user_id=f65910b1bd7b4a1f8dd064a65c91f102, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.666162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.678000 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1391/2770] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:29 2022] PUT /identity/v3/projects/2fb0eb9a63b149f49cfd078eff2f17bf/users/f65910b1bd7b4a1f8dd064a65c91f102/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:29:29.688203 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.688490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.688765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.718262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] MFA Rules not processed for user `f65910b1bd7b4a1f8dd064a65c91f102`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:29.719585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:29.792891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1380/2771] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:29.804559 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.806817 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:29.830479 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:29.867254 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe3b79-e117-4e61-8a05-038a57310538 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.871441 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.872135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.872419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.872692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:29.883605 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:29.885058 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:29.895094 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1392/2772] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:29 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:29.972868 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:29.983729 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efc22b-4330-46c5-8756-3e7eb7a4db1b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:29.987442 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329f687-4a8d-4fa0-bf9d-88b1ff74e9ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:29.988097 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329f687-4a8d-4fa0-bf9d-88b1ff74e9ce service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:29.988377 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329f687-4a8d-4fa0-bf9d-88b1ff74e9ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:29.988644 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329f687-4a8d-4fa0-bf9d-88b1ff74e9ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:30.047218 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329f687-4a8d-4fa0-bf9d-88b1ff74e9ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:30.048670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329f687-4a8d-4fa0-bf9d-88b1ff74e9ce service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:30.061903 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1381/2773] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:29 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:39.890473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=17 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:40.053886 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=88 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:45.022961 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:45.093319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf83ac-f4bf-43bd-aabd-61c503edb5dc service cinder] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:45.099650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:45.101078 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:45.101558 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:45.102052 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] PATH_INFO: `/v3/users/f53f617dcc5d4b1a966989ef3dea5075` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:45.104214 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53f617dcc5d4b1a966989ef3dea5075)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:45.107050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:45.162359 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53f617dcc5d4b1a966989ef3dea5075'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.163029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53f617dcc5d4b1a966989ef3dea5075'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.171364 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53f617dcc5d4b1a966989ef3dea5075'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.172095 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53f617dcc5d4b1a966989ef3dea5075'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.185863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53f617dcc5d4b1a966989ef3dea5075'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.201144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1393/2774] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:45 2022] DELETE /identity/v3/users/f53f617dcc5d4b1a966989ef3dea5075 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:45.205689 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:45.206209 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:45.206318 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:45.223934 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:45.225176 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:45.286169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1382/2775] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:45.296527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:45.298105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:45.360826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e115a-8f6f-4a13-a02f-2c35dc309242 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:45.365171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:45.365831 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:45.366150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:45.366432 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:45.376858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:45.378321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:45.388540 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1394/2776] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:45.687996 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:45.689691 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:45.702289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aee28-3fec-4eb9-bb06-2254f4d228c6 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:45.712439 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:45.713308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:45.713591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:45.713858 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] PATH_INFO: `/v3/projects/47bab14623e646acb4f676dae1107ead` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:45.715153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bab14623e646acb4f676dae1107ead)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:45.717110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:45.773961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bab14623e646acb4f676dae1107ead', 'request_id': 'req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.781971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bab14623e646acb4f676dae1107ead', 'request_id': 'req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.791003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bab14623e646acb4f676dae1107ead', 'request_id': 'req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.802784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] The token cache is being invalidate because project 47bab14623e646acb4f676dae1107ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:45.803079 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.803598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.805143 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1383/2777] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:45 2022] DELETE /identity/v3/projects/47bab14623e646acb4f676dae1107ead => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:45.809414 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:45.819735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35df08b-8d68-433b-90cd-67c7d76eb12d service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:45.839117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:45.881166 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:45.882047 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:45.882331 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:45.882635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] PATH_INFO: `/v3/users/f65910b1bd7b4a1f8dd064a65c91f102` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:45.884380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65910b1bd7b4a1f8dd064a65c91f102)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:45.886316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:45.940787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65910b1bd7b4a1f8dd064a65c91f102'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.941577 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65910b1bd7b4a1f8dd064a65c91f102'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.949155 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65910b1bd7b4a1f8dd064a65c91f102'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.949815 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65910b1bd7b4a1f8dd064a65c91f102'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.962634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-0de4f8b4-613e-4e64-a17d-2b82ce550aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65910b1bd7b4a1f8dd064a65c91f102'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:45.977953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1395/2778] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:45 2022] DELETE /identity/v3/users/f65910b1bd7b4a1f8dd064a65c91f102 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:46.263160 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.265050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:46.285311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f4d84-c875-4a5f-b1ee-d411dd9493b4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.349386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.350167 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.350451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.350727 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] PATH_INFO: `/v3/projects/2fb0eb9a63b149f49cfd078eff2f17bf` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.351937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb0eb9a63b149f49cfd078eff2f17bf)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.354445 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.385568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:46.404842 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb0eb9a63b149f49cfd078eff2f17bf', 'request_id': 'req-6e81aa69-f169-4e7d-971b-6afb4555e598'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:46.411255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb0eb9a63b149f49cfd078eff2f17bf', 'request_id': 'req-6e81aa69-f169-4e7d-971b-6afb4555e598'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:46.419964 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb0eb9a63b149f49cfd078eff2f17bf', 'request_id': 'req-6e81aa69-f169-4e7d-971b-6afb4555e598'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:46.434172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] The token cache is being invalidate because project 2fb0eb9a63b149f49cfd078eff2f17bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:46.434512 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:46.435219 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-6e81aa69-f169-4e7d-971b-6afb4555e598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:46.436861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1384/2779] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:46 2022] DELETE /identity/v3/projects/2fb0eb9a63b149f49cfd078eff2f17bf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:46.443243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.443770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.443770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.465648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:46.467366 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:46.534302 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1396/2780] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:46.543419 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.543716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.544010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.563583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:46.564859 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:46.573174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1385/2781] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:46.577640 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.588845 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd1a45-8deb-4853-8c04-10d778366c0b None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.593503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.594173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.594442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.594722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.595485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:29:46.595797 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.597397 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.610158 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1397/2782] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:29:46.617645 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.628924 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759a3b8-14fd-430c-a3be-3cdf9b111b7b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.634424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d3960-9116-488b-af8c-957368676976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.634898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3960-9116-488b-af8c-957368676976 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.635188 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3960-9116-488b-af8c-957368676976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.635467 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3960-9116-488b-af8c-957368676976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.636161 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d3960-9116-488b-af8c-957368676976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.638093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d3960-9116-488b-af8c-957368676976 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.668373 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1386/2783] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:29:46 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:46.672884 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.683857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd759a-45bf-4df4-98bb-982077c620df admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.712578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:46.750256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.751049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.751381 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.751857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.752740 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.755112 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.781485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1398/2784] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:29:46 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:46.785647 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d3960-9116-488b-af8c-957368676976 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.796684 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d3960-9116-488b-af8c-957368676976 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.802267 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.802961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.803249 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.803529 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.804270 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.805887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.818677 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1387/2785] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:46.823384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.834309 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3452e9d-0976-44a3-8b69-675501391759 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.839799 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7de3ad-95d7-4b47-abf3-081f135b0af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.840487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7de3ad-95d7-4b47-abf3-081f135b0af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.840773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7de3ad-95d7-4b47-abf3-081f135b0af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.841082 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7de3ad-95d7-4b47-abf3-081f135b0af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.841829 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7de3ad-95d7-4b47-abf3-081f135b0af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.843549 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7de3ad-95d7-4b47-abf3-081f135b0af7 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.854245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1399/2786] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:46.858649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:46.868721 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36702c7d-7da2-4996-bd2a-90a2329b5824 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:46.873307 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:46.874197 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.874480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.874758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] PATH_INFO: `/v3/projects/9c8b82d7186440fc9c68c09dc526ee6e/users/56172dd0e77a4c1fac0a023e56926076/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.891213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8b82d7186440fc9c68c09dc526ee6e, user_id=56172dd0e77a4c1fac0a023e56926076, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:46.894102 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:46.908000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1388/2787] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:46 2022] PUT /identity/v3/projects/9c8b82d7186440fc9c68c09dc526ee6e/users/56172dd0e77a4c1fac0a023e56926076/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:46.918487 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:46.918776 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:46.919093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:46.947787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] MFA Rules not processed for user `56172dd0e77a4c1fac0a023e56926076`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:46.949228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:47.017130 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1400/2788] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:47.021750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.085362 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf96d3-6ba8-4e55-8565-4789c79c3ca1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.090150 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.090806 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.091126 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.091409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.092110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.094011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.122430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1389/2789] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:29:47 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:47.126465 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.138077 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4d708-5da2-4496-91eb-d7d7307fea51 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.196663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.197330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.197642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.197912 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.198594 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.200554 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.221984 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1401/2790] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:29:47 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:47.226796 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.237478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89747f-1f48-467a-b40f-161db1aa9a19 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.242122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.243288 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.243569 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.243843 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.244553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.246134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.256755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1390/2791] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:29:47.261483 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.272236 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f1386-f123-46d9-b9fa-11d3d6982bdd admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.280548 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.281431 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.281713 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.281986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] PATH_INFO: `/v3/projects/b95f9011028e477fb1baacfd9704f1ea/users/51b8bc4ca9ee4d5698a4cf11071b29e1/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.297344 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95f9011028e477fb1baacfd9704f1ea, user_id=51b8bc4ca9ee4d5698a4cf11071b29e1, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.300039 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.314108 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1402/2792] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:47 2022] PUT /identity/v3/projects/b95f9011028e477fb1baacfd9704f1ea/users/51b8bc4ca9ee4d5698a4cf11071b29e1/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:47.318758 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.330535 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10e78-5e6f-4d56-b833-e2a75f7b5b61 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.351728 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:47.391121 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15346fd5-1af3-4e30-872b-838cf9dd2c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.391706 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15346fd5-1af3-4e30-872b-838cf9dd2c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.391991 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15346fd5-1af3-4e30-872b-838cf9dd2c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.392258 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15346fd5-1af3-4e30-872b-838cf9dd2c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.392977 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15346fd5-1af3-4e30-872b-838cf9dd2c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.394881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15346fd5-1af3-4e30-872b-838cf9dd2c88 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.405338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1391/2793] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:29:47.409698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.415648 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:47.422494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a854a-5cba-42ea-a85e-bec9b9a994ff admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.428008 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.428905 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.429180 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.429451 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] PATH_INFO: `/v3/projects/b95f9011028e477fb1baacfd9704f1ea/users/51b8bc4ca9ee4d5698a4cf11071b29e1/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.432071 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b95f9011028e477fb1baacfd9704f1ea, user_id=51b8bc4ca9ee4d5698a4cf11071b29e1, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.434735 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.445858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1403/2794] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:47 2022] PUT /identity/v3/projects/b95f9011028e477fb1baacfd9704f1ea/users/51b8bc4ca9ee4d5698a4cf11071b29e1/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:47.457030 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.457324 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.457602 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.486277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] MFA Rules not processed for user `51b8bc4ca9ee4d5698a4cf11071b29e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:47.487534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:47.562220 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1392/2795] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:47.573687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:47.575751 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:47.652106 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ed8a8-0ab9-494a-8c41-ad906ecfc7e8 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:47.655946 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e3357-8479-4e76-a8af-3590d4971d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:47.656615 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e3357-8479-4e76-a8af-3590d4971d17 service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:47.656918 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e3357-8479-4e76-a8af-3590d4971d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:47.657201 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e3357-8479-4e76-a8af-3590d4971d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:47.671256 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e3357-8479-4e76-a8af-3590d4971d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:47.672787 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e3357-8479-4e76-a8af-3590d4971d17 service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:47.685920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1404/2796] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:49.156798 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:49.232210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba75ff-18db-419f-8bc3-8152b21062e7 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:49.237036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:49.237860 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:49.238148 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:49.238423 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] PATH_INFO: `/v3/users/56172dd0e77a4c1fac0a023e56926076` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:49.239668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56172dd0e77a4c1fac0a023e56926076)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:49.241600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:49.299331 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56172dd0e77a4c1fac0a023e56926076'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:49.300212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56172dd0e77a4c1fac0a023e56926076'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:49.301436 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:49.309959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56172dd0e77a4c1fac0a023e56926076'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:49.311168 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56172dd0e77a4c1fac0a023e56926076'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:49.324586 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56172dd0e77a4c1fac0a023e56926076'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:49.349959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1393/2797] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:49 2022] DELETE /identity/v3/users/56172dd0e77a4c1fac0a023e56926076 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:29:49.355601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:49.356137 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:49.356857 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:49.377771 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:49.379654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:49.459012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1405/2798] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:49.470192 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:49.472400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:49.554791 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290bd74-cf0c-44f0-b4a4-efa56ba44da3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:49.558431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:49.559106 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:49.559391 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:49.559665 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:49.569303 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:49.570786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:49.580970 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1394/2799] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:49.909793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:49.913475 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:49.928653 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109fef6-df30-4025-8143-1cddf42be15d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:49.933442 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:49.934333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:49.934588 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:49.934956 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] PATH_INFO: `/v3/projects/9c8b82d7186440fc9c68c09dc526ee6e` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:49.936114 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8b82d7186440fc9c68c09dc526ee6e)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:49.938015 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:49.989638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8b82d7186440fc9c68c09dc526ee6e', 'request_id': 'req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.001333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8b82d7186440fc9c68c09dc526ee6e', 'request_id': 'req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.007255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8b82d7186440fc9c68c09dc526ee6e', 'request_id': 'req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.018923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] The token cache is being invalidate because project 9c8b82d7186440fc9c68c09dc526ee6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:50.019212 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.019861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.021409 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1406/2800] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:49 2022] DELETE /identity/v3/projects/9c8b82d7186440fc9c68c09dc526ee6e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:50.025537 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:50.035576 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd4e3a-da11-4959-8712-3a093160437a service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:50.107085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:50.107864 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.108145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.108418 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] PATH_INFO: `/v3/users/51b8bc4ca9ee4d5698a4cf11071b29e1` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.109633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b8bc4ca9ee4d5698a4cf11071b29e1)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:50.111589 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:50.160216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b8bc4ca9ee4d5698a4cf11071b29e1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.160935 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b8bc4ca9ee4d5698a4cf11071b29e1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.168278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b8bc4ca9ee4d5698a4cf11071b29e1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.168915 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b8bc4ca9ee4d5698a4cf11071b29e1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.179633 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-4ce37783-96c9-4967-ac95-926119979257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b8bc4ca9ee4d5698a4cf11071b29e1'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.191411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:50.209103 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1395/2801] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:29:50 2022] DELETE /identity/v3/users/51b8bc4ca9ee4d5698a4cf11071b29e1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:50.496646 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:50.498120 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:50.511428 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536b1c8-c6ca-4e06-ac70-615fbfffe01a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:50.526140 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:50.579145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:50.580169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.580698 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.581229 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] PATH_INFO: `/v3/projects/b95f9011028e477fb1baacfd9704f1ea` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.582710 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b95f9011028e477fb1baacfd9704f1ea)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:50.584978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:50.638670 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b95f9011028e477fb1baacfd9704f1ea', 'request_id': 'req-ef498d28-487c-4cf0-abcb-7eba282bd7a0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.648242 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b95f9011028e477fb1baacfd9704f1ea', 'request_id': 'req-ef498d28-487c-4cf0-abcb-7eba282bd7a0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.654240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b95f9011028e477fb1baacfd9704f1ea', 'request_id': 'req-ef498d28-487c-4cf0-abcb-7eba282bd7a0'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.666128 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] The token cache is being invalidate because project b95f9011028e477fb1baacfd9704f1ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:29:50.666502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.667127 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-ef498d28-487c-4cf0-abcb-7eba282bd7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:29:50.668690 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1407/2802] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:29:50 2022] DELETE /identity/v3/projects/b95f9011028e477fb1baacfd9704f1ea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:50.673755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.674101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.674424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.695861 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:50.697129 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:50.759451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1396/2803] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:50.767973 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.768476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.768878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.791286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:50.792513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:50.800669 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1408/2804] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:50.805011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:50.815231 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a091179-0a03-4ca2-88c2-2200ebd512a1 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:50.821290 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:50.822045 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.822367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.822670 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.823540 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:29:50.823891 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:50.825848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:50.840152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1397/2805] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:29:50.844657 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:50.857542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a25ae-887b-4a8a-b738-f7db43d4ae06 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:50.862430 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:50.863286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.863607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.864087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.864913 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:50.867362 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:50.895467 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1409/2806] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:29:50 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:50.900463 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:50.910618 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449aa2-b178-4335-a7a9-de3869b9f772 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:50.969289 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:50.970515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:50.970515 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:50.970741 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:50.971568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:50.973676 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:50.999325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1398/2807] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:29:50 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:51.003689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.014438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ab4f-954a-4a6f-8ebe-fcbca42beed4 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.019570 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.020247 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.020534 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.020809 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.021542 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.023175 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.034321 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1410/2808] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:51.039020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.053916 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd06ca-1ad1-47e0-9a4f-3b7aef9afff6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.059595 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f70b77-9fcb-47cc-81f4-54cbec27137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.060274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f70b77-9fcb-47cc-81f4-54cbec27137a admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.060560 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f70b77-9fcb-47cc-81f4-54cbec27137a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.060833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f70b77-9fcb-47cc-81f4-54cbec27137a admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.061597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f70b77-9fcb-47cc-81f4-54cbec27137a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.063238 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f70b77-9fcb-47cc-81f4-54cbec27137a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.073766 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1399/2809] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:51.078068 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.089811 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d883c5-eb91-4b4d-b7c1-e2b14e279d7c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.094978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.095989 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.096368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.096737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] PATH_INFO: `/v3/projects/6c3a4d40546f4bab98daabaed8cbbabd/users/d287186a3ea64fc398906a0f76357b3e/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.113967 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c3a4d40546f4bab98daabaed8cbbabd, user_id=d287186a3ea64fc398906a0f76357b3e, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.116650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.131145 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1411/2810] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:51 2022] PUT /identity/v3/projects/6c3a4d40546f4bab98daabaed8cbbabd/users/d287186a3ea64fc398906a0f76357b3e/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:51.141887 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.142182 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.142493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.171753 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] MFA Rules not processed for user `d287186a3ea64fc398906a0f76357b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:51.172877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:51.217049 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:51.244005 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1400/2811] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:29:51.248393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.294100 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:51.313793 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df2d78-59fe-4c9b-b5c0-b226fe376815 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.318687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.319373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.319659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.319962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.320665 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.322566 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.355105 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1412/2812] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:29:51 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:51.359680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.375061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97b946-e569-44c7-ad22-05c7c58cee1b None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.444151 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.444919 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.445206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.445483 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.446179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.448133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.468655 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1401/2813] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:29:51 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:29:51.472672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.484536 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d438-bec8-406c-9b7d-429190fc624a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.489287 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.489957 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.490245 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.490526 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.491324 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.493593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.505641 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1413/2814] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:29:51.509865 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.520256 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb46fd-6f1f-4de3-8fea-48a925d995a1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.525513 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.526400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.526680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.526989 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] PATH_INFO: `/v3/projects/fc714d181d1c48829879c7e674bd6dd6/users/295ef6becc3a4109bd64b9fa274109a5/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.542902 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc714d181d1c48829879c7e674bd6dd6, user_id=295ef6becc3a4109bd64b9fa274109a5, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.545556 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.560375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1402/2815] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:51 2022] PUT /identity/v3/projects/fc714d181d1c48829879c7e674bd6dd6/users/295ef6becc3a4109bd64b9fa274109a5/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:51.564642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.575012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac7e9-2272-4ef8-8ad8-0c4b736feb59 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.634014 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c34da8-689a-43bb-8206-287e97d23453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.634687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34da8-689a-43bb-8206-287e97d23453 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.634999 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34da8-689a-43bb-8206-287e97d23453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.635275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c34da8-689a-43bb-8206-287e97d23453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.636006 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34da8-689a-43bb-8206-287e97d23453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.637607 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c34da8-689a-43bb-8206-287e97d23453 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.647739 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1414/2816] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:29:51.652788 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.664240 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b247096-cc0a-463e-bc1e-36d4933a3b8b admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.669771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.670697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.671010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.671286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] PATH_INFO: `/v3/projects/fc714d181d1c48829879c7e674bd6dd6/users/295ef6becc3a4109bd64b9fa274109a5/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.673306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc714d181d1c48829879c7e674bd6dd6, user_id=295ef6becc3a4109bd64b9fa274109a5, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.676453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:51.687421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1403/2817] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:29:51 2022] PUT /identity/v3/projects/fc714d181d1c48829879c7e674bd6dd6/users/295ef6becc3a4109bd64b9fa274109a5/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:29:51.751539 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.751832 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.752115 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.779678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] MFA Rules not processed for user `295ef6becc3a4109bd64b9fa274109a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:29:51.780922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:51.856996 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1415/2818] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:29:51.867744 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:51.869547 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:29:51.933981 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f35e9-0d15-4122-9f27-40a64fa90d9d admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:51.937710 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:51.938291 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:51.938571 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:51.938870 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:51.986210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:29:51.996855 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:51.998300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:52.007501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1404/2819] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:29:56.660861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:29:56.670778 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a774855-b72c-4ea1-9e24-a79fac88e084 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:29:56.674370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bc3c6-483a-4ec0-b4b0-691ed2af1bcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:29:56.675049 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc3c6-483a-4ec0-b4b0-691ed2af1bcf service cinder] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:29:56.675333 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc3c6-483a-4ec0-b4b0-691ed2af1bcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:29:56.675609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bc3c6-483a-4ec0-b4b0-691ed2af1bcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:29:56.685638 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bc3c6-483a-4ec0-b4b0-691ed2af1bcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:29:56.687043 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592bc3c6-483a-4ec0-b4b0-691ed2af1bcf service cinder] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:29:56.697910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1416/2820] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:29:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:02.003248 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:06.692782 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:19.557558 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:19.668344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc94e22-7c55-4d2f-98cb-c011dbe78060 service cinder] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:19.673054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:19.673917 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:19.674195 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:19.674465 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] PATH_INFO: `/v3/users/d287186a3ea64fc398906a0f76357b3e` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:19.675680 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d287186a3ea64fc398906a0f76357b3e)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:19.677599 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:19.740264 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd287186a3ea64fc398906a0f76357b3e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:19.745266 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd287186a3ea64fc398906a0f76357b3e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:19.751311 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd287186a3ea64fc398906a0f76357b3e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:19.752679 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd287186a3ea64fc398906a0f76357b3e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:19.762399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd287186a3ea64fc398906a0f76357b3e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:19.781604 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1405/2821] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:30:19 2022] DELETE /identity/v3/users/d287186a3ea64fc398906a0f76357b3e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:19.786341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:19.786889 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:19.787036 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:19.848437 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:19.849681 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:19.916597 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1417/2822] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:30:19.926368 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:19.937367 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:19.997257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:20.061273 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9089fe-a7c2-4152-85ec-83900ce86cd5 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:20.065616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:20.066124 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:20.066405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:20.066683 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:20.076709 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:20.078201 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:20.089310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1406/2823] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:20.416617 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:20.418368 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:20.431283 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70627510-5fe4-4c95-a16d-12219a606a4f None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:20.438258 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:20.438715 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:20.439037 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:20.439316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] PATH_INFO: `/v3/projects/6c3a4d40546f4bab98daabaed8cbbabd` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:20.440561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c3a4d40546f4bab98daabaed8cbbabd)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:20.442546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:20.505914 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c3a4d40546f4bab98daabaed8cbbabd', 'request_id': 'req-fc521189-21a3-43f9-ad7c-da29664c68ca'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.515923 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c3a4d40546f4bab98daabaed8cbbabd', 'request_id': 'req-fc521189-21a3-43f9-ad7c-da29664c68ca'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.523413 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c3a4d40546f4bab98daabaed8cbbabd', 'request_id': 'req-fc521189-21a3-43f9-ad7c-da29664c68ca'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.537104 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] The token cache is being invalidate because project 6c3a4d40546f4bab98daabaed8cbbabd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:30:20.537627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.538593 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.541144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1418/2824] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:30:20 2022] DELETE /identity/v3/projects/6c3a4d40546f4bab98daabaed8cbbabd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:20.545972 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:20.557881 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2d42-075d-47df-b9a9-2b7974a981b2 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:20.624481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:20.625403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:20.625687 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:20.625959 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] PATH_INFO: `/v3/users/295ef6becc3a4109bd64b9fa274109a5` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:20.627356 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=295ef6becc3a4109bd64b9fa274109a5)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:20.629308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:20.679227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '295ef6becc3a4109bd64b9fa274109a5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.680149 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '295ef6becc3a4109bd64b9fa274109a5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.691174 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '295ef6becc3a4109bd64b9fa274109a5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.692157 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '295ef6becc3a4109bd64b9fa274109a5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.702169 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-67f4300a-f6cd-488e-8aee-68afb9523efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '295ef6becc3a4109bd64b9fa274109a5'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:20.727475 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1407/2825] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:30:20 2022] DELETE /identity/v3/users/295ef6becc3a4109bd64b9fa274109a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:21.019181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.020769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:21.035445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc521189-21a3-43f9-ad7c-da29664c68ca admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.088865 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:21.096675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.097620 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.098135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.098438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] PATH_INFO: `/v3/projects/fc714d181d1c48829879c7e674bd6dd6` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.099896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc714d181d1c48829879c7e674bd6dd6)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.102380 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.149340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc714d181d1c48829879c7e674bd6dd6', 'request_id': 'req-2cd974ae-ec32-4f07-90c9-83959333f0b8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:21.157353 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc714d181d1c48829879c7e674bd6dd6', 'request_id': 'req-2cd974ae-ec32-4f07-90c9-83959333f0b8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:21.163628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc714d181d1c48829879c7e674bd6dd6', 'request_id': 'req-2cd974ae-ec32-4f07-90c9-83959333f0b8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:21.174687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] The token cache is being invalidate because project fc714d181d1c48829879c7e674bd6dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80507) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:30:21.175010 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:21.175644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-2cd974ae-ec32-4f07-90c9-83959333f0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:21.177228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1419/2826] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:30:21 2022] DELETE /identity/v3/projects/fc714d181d1c48829879c7e674bd6dd6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:21.183163 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.183454 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.183732 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.215415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:21.235122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:21.236750 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:21.299624 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1408/2827] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:21.307825 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.308143 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.308427 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.327185 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:21.328408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:21.337827 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1420/2828] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:21.342490 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.354059 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0eec4-9697-47e0-a394-dccf1009bbeb None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.358966 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.359688 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.359968 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.360241 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.360971 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80506) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:30:21.361282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.362912 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.375495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1409/2829] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:30:21.380163 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.389358 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06b305-33ab-4143-8700-ab4fe8c21b07 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.394215 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.394888 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.395173 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.395444 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.396135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.398033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.424072 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1421/2830] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:21.428190 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.439437 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841937e-0140-4e59-a5df-074334d10694 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.500430 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.501091 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.501405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.501720 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.502415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.504366 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.527312 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1410/2831] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:21.531578 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.542724 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5818-9b6e-49e9-a456-f792e241ec64 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.547532 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.548253 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.548654 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.548864 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.549702 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.551529 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.563367 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1422/2832] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:21.567464 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.583206 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137b1fe-ce73-460a-9fc9-12349ec44198 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.589039 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495a5b6-8054-4955-bfd6-9d74551d8ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.589763 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a5b6-8054-4955-bfd6-9d74551d8ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.590100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a5b6-8054-4955-bfd6-9d74551d8ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.590379 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495a5b6-8054-4955-bfd6-9d74551d8ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.591286 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495a5b6-8054-4955-bfd6-9d74551d8ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.592948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495a5b6-8054-4955-bfd6-9d74551d8ee3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.604642 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1411/2833] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:21.609048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.619268 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afafea0-3a54-4bd0-9b74-2186e88d912a admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.624167 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.625050 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.625385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.626219 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] PATH_INFO: `/v3/projects/4e0df75c25304d3880c6dab99d59ff7a/users/cc31b3c3939c4616a6f4090bb4175f5c/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.641826 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0df75c25304d3880c6dab99d59ff7a, user_id=cc31b3c3939c4616a6f4090bb4175f5c, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.644920 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.658384 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1423/2834] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:21 2022] PUT /identity/v3/projects/4e0df75c25304d3880c6dab99d59ff7a/users/cc31b3c3939c4616a6f4090bb4175f5c/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:21.670054 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.670453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.670735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.709591 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] MFA Rules not processed for user `cc31b3c3939c4616a6f4090bb4175f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:21.711344 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:21.769814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:21.780132 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1412/2835] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:21.784575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.851304 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bba1a-9ef2-42dd-aaa8-a8a4c904d3bc admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.856449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.857103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.857387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.857662 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.858382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.860314 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.885113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1424/2836] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:21.889216 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.900269 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c6305-96c8-423a-a02d-e6d5913c9607 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.955613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.956278 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.956568 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.956854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:21.957553 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:21.959523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:21.979338 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1413/2837] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:30:21 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:21.983545 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:21.993769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505b020-b744-42a4-90ec-44f820db4e06 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:21.998393 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:21.999087 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:21.999374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:21.999650 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.000371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.001951 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.003953 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:22.014683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1425/2838] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:22.018551 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:22.029613 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2055694-19f0-4df5-a9c8-ac419a00c5b2 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:22.034213 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:22.035120 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.035403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.035675 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] PATH_INFO: `/v3/projects/ba2af8374fdc411ebe623baf89b58f81/users/45c31c047524476a9c0749648bdc93cc/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.050782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2af8374fdc411ebe623baf89b58f81, user_id=45c31c047524476a9c0749648bdc93cc, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.053481 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.066131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1414/2839] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:22 2022] PUT /identity/v3/projects/ba2af8374fdc411ebe623baf89b58f81/users/45c31c047524476a9c0749648bdc93cc/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:22.070671 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:22.080612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ca32d-cf29-4439-b011-c79d5e71c6cf admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:22.135557 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70ee60-30a0-4a26-b3be-10ce7076ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:22.136142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ee60-30a0-4a26-b3be-10ce7076ecd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.136419 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ee60-30a0-4a26-b3be-10ce7076ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.136689 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70ee60-30a0-4a26-b3be-10ce7076ecd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.137408 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ee60-30a0-4a26-b3be-10ce7076ecd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.139012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70ee60-30a0-4a26-b3be-10ce7076ecd6 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.149357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1426/2840] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 07 13:30:22.153501 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:22.163738 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826be4e-423d-43d4-8041-9e947f88bacb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:22.168370 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:22.169239 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.169528 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.169812 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] PATH_INFO: `/v3/projects/ba2af8374fdc411ebe623baf89b58f81/users/45c31c047524476a9c0749648bdc93cc/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.171821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2af8374fdc411ebe623baf89b58f81, user_id=45c31c047524476a9c0749648bdc93cc, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.174470 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.184434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1415/2841] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:22 2022] PUT /identity/v3/projects/ba2af8374fdc411ebe623baf89b58f81/users/45c31c047524476a9c0749648bdc93cc/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:22.194026 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.194861 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.195147 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.222238 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] MFA Rules not processed for user `45c31c047524476a9c0749648bdc93cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:22.223472 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:22.294510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1427/2842] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:22.306255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:22.308048 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:22.376893 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafebb84-f446-462b-b289-5a6b7c2592a6 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:22.380672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075aa06-1330-4275-906d-b23ad1a428a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:22.381294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075aa06-1330-4275-906d-b23ad1a428a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.381578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075aa06-1330-4275-906d-b23ad1a428a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.382484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075aa06-1330-4275-906d-b23ad1a428a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.391840 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075aa06-1330-4275-906d-b23ad1a428a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.393180 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075aa06-1330-4275-906d-b23ad1a428a3 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.402628 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1416/2843] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:22.471707 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:22.541863 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0938e-6b3e-4d3b-b859-040673e23fd0 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:22.546678 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:22.547586 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.547872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.548150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] PATH_INFO: `/v3/users/cc31b3c3939c4616a6f4090bb4175f5c` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.549383 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc31b3c3939c4616a6f4090bb4175f5c)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.551348 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.574171 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:22.598502 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc31b3c3939c4616a6f4090bb4175f5c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:22.599169 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc31b3c3939c4616a6f4090bb4175f5c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:22.606891 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc31b3c3939c4616a6f4090bb4175f5c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:22.607527 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc31b3c3939c4616a6f4090bb4175f5c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:22.620341 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc31b3c3939c4616a6f4090bb4175f5c'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:22.639162 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1428/2844] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:30:22 2022] DELETE /identity/v3/users/cc31b3c3939c4616a6f4090bb4175f5c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 07 13:30:22.643851 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.644321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.644321 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.665597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:22.666821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:22.735476 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:22.754002 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1417/2845] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:22.765191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:22.766610 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:22.823464 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8ef09-6420-4c47-9f38-488faa444fe2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:22.827144 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:22.827785 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:22.828066 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:22.828342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:22.837928 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:22.839395 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:22.848773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1429/2846] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:23.122536 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:23.124325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:23.138473 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1aa0e-ce45-485d-864a-363dd6303ed4 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:23.143931 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:23.144822 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:23.145061 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:23.145333 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] PATH_INFO: `/v3/projects/4e0df75c25304d3880c6dab99d59ff7a` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:23.146517 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0df75c25304d3880c6dab99d59ff7a)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:23.148493 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:23.195199 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0df75c25304d3880c6dab99d59ff7a', 'request_id': 'req-9196882e-0ed4-4732-a726-16ef3c64c91a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.201330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0df75c25304d3880c6dab99d59ff7a', 'request_id': 'req-9196882e-0ed4-4732-a726-16ef3c64c91a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.208664 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0df75c25304d3880c6dab99d59ff7a', 'request_id': 'req-9196882e-0ed4-4732-a726-16ef3c64c91a'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.219162 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] The token cache is being invalidate because project 4e0df75c25304d3880c6dab99d59ff7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:30:23.219451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.220085 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.221652 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1418/2847] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:30:23 2022] DELETE /identity/v3/projects/4e0df75c25304d3880c6dab99d59ff7a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:23.225688 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:23.235517 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24138f2-e5d8-46cb-88e5-45fd391b0337 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:23.290731 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:23.291635 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:23.291919 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:23.292195 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] PATH_INFO: `/v3/users/45c31c047524476a9c0749648bdc93cc` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:23.293387 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c31c047524476a9c0749648bdc93cc)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:23.295330 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:23.340152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:23.343396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c31c047524476a9c0749648bdc93cc'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.344300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c31c047524476a9c0749648bdc93cc'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.353228 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c31c047524476a9c0749648bdc93cc'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.353858 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c31c047524476a9c0749648bdc93cc'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.366568 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-a9562209-4255-4598-9884-bca9925cadf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c31c047524476a9c0749648bdc93cc'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.381135 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1430/2848] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:30:23 2022] DELETE /identity/v3/users/45c31c047524476a9c0749648bdc93cc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:23.646579 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:23.648253 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:23.672000 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196882e-0ed4-4732-a726-16ef3c64c91a admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:23.733948 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:23.735021 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:23.735400 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:23.735703 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] PATH_INFO: `/v3/projects/ba2af8374fdc411ebe623baf89b58f81` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:23.737008 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2af8374fdc411ebe623baf89b58f81)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:23.738949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:23.785552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2af8374fdc411ebe623baf89b58f81', 'request_id': 'req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.792212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2af8374fdc411ebe623baf89b58f81', 'request_id': 'req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.799375 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2af8374fdc411ebe623baf89b58f81', 'request_id': 'req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.809672 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] The token cache is being invalidate because project ba2af8374fdc411ebe623baf89b58f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:30:23.809961 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.810587 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-f27b9736-e7c7-4f45-9939-4f3fc1a2b410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:23.812233 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1419/2849] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:30:23 2022] DELETE /identity/v3/projects/ba2af8374fdc411ebe623baf89b58f81 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:23.817546 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:23.817839 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:23.818152 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:23.837734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:23.838979 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:23.904294 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1431/2850] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:23.912133 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:23.912422 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:23.912752 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:23.920937 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:23.935833 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:23.937300 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:23.945649 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1420/2851] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:23.951744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:23.965211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52054e6-815c-48b2-92aa-249b8b84019d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:23.969798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:23.970449 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:23.970726 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:23.971029 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:23.971757 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:30:23.972064 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:23.973655 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:23.986188 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1432/2852] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 07 13:30:23.990789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:24.000361 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09662f-fc45-413d-b297-ec359d561eea None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:24.005205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:24.005927 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.006282 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.006594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.007394 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:24.009386 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:24.035179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1421/2853] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:30:23 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:24.040235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:24.050490 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e5d1b-2bff-4e6d-a9d7-684c7e020bb9 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:24.107708 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:24.108371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.108656 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.108934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.109628 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:24.111609 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:24.130683 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1433/2854] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:30:24 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:24.135962 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:24.148179 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b2dd-48ad-47c9-9053-31c0402f9db4 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:24.152814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:24.153480 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.153771 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.154050 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.154813 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:24.156462 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:24.167277 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1422/2855] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:24.171301 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:24.181389 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827fbc0-7cde-4145-900c-7e2734b57c97 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:24.186089 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de55f20-0c2a-4885-b264-05b339adf60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:24.186760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de55f20-0c2a-4885-b264-05b339adf60f admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.187079 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de55f20-0c2a-4885-b264-05b339adf60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.187357 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de55f20-0c2a-4885-b264-05b339adf60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.188093 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de55f20-0c2a-4885-b264-05b339adf60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:24.189719 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de55f20-0c2a-4885-b264-05b339adf60f admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:24.200473 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1434/2856] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:24.204768 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:24.215637 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba3230-519f-4751-ac47-af379f832933 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:24.220495 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:24.221365 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.221646 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.221920 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] PATH_INFO: `/v3/projects/00e2faff80764bc0ad441cb80a8051fc/users/18cb1622f4a145b9871a36895c68e2fd/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.237482 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e2faff80764bc0ad441cb80a8051fc, user_id=18cb1622f4a145b9871a36895c68e2fd, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:24.240251 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:24.256414 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1423/2857] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:24 2022] PUT /identity/v3/projects/00e2faff80764bc0ad441cb80a8051fc/users/18cb1622f4a145b9871a36895c68e2fd/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:24.269004 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.269506 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.269626 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.296644 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] MFA Rules not processed for user `18cb1622f4a145b9871a36895c68e2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:24.297856 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:24.310269 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:24.364575 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1435/2858] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:24.375735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:24.377578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:24.438975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34554f-eef4-440c-8ee7-d76a45ef98dc admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:24.442638 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532244d7-1b5c-44fb-9390-bbe1632fa180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27c81d5738744687a915940977f0f647', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:24.443310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532244d7-1b5c-44fb-9390-bbe1632fa180 service cinder] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:24.443596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532244d7-1b5c-44fb-9390-bbe1632fa180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:24.443872 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532244d7-1b5c-44fb-9390-bbe1632fa180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:24.452719 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532244d7-1b5c-44fb-9390-bbe1632fa180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:24.454152 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532244d7-1b5c-44fb-9390-bbe1632fa180 service cinder] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:24.463737 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1424/2859] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:32.041784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:32.103998 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a273-6601-4bf3-81af-f54107c30cdd None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:32.108760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:32.109625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.109910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.110191 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] PATH_INFO: `/v3/users/18cb1622f4a145b9871a36895c68e2fd` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.111392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18cb1622f4a145b9871a36895c68e2fd)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:32.113306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:32.159764 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18cb1622f4a145b9871a36895c68e2fd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.160422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18cb1622f4a145b9871a36895c68e2fd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.167976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18cb1622f4a145b9871a36895c68e2fd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.168606 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18cb1622f4a145b9871a36895c68e2fd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.180423 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18cb1622f4a145b9871a36895c68e2fd'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.193123 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1436/2860] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:30:32 2022] DELETE /identity/v3/users/18cb1622f4a145b9871a36895c68e2fd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:32.197696 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.198205 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.198308 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.215616 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:32.216821 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:32.284294 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1425/2861] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:32.294915 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:32.298571 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:32.341257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:32.360760 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ebd1-2fd2-4253-8f96-5427cd91a238 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:32.364509 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4456695-cf51-4bf5-91e3-7b19bff81dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:32.365117 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4456695-cf51-4bf5-91e3-7b19bff81dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.365396 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4456695-cf51-4bf5-91e3-7b19bff81dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.365666 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4456695-cf51-4bf5-91e3-7b19bff81dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.374769 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4456695-cf51-4bf5-91e3-7b19bff81dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:32.376217 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4456695-cf51-4bf5-91e3-7b19bff81dfb service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:32.385966 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1437/2862] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:32.672523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:32.674469 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:32.691717 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3c49-f8b9-4ec4-b264-99dec9a693ad None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:32.697109 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:32.698073 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.698304 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.698524 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] PATH_INFO: `/v3/projects/00e2faff80764bc0ad441cb80a8051fc` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.699938 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e2faff80764bc0ad441cb80a8051fc)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:32.701759 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:32.746867 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:32.754901 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e2faff80764bc0ad441cb80a8051fc', 'request_id': 'req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.762342 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e2faff80764bc0ad441cb80a8051fc', 'request_id': 'req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.770153 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e2faff80764bc0ad441cb80a8051fc', 'request_id': 'req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.784246 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] The token cache is being invalidate because project 00e2faff80764bc0ad441cb80a8051fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:30:32.784541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.785212 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-16dbddb4-d82d-4ec2-8f2c-349c513b66ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:30:32.787093 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1426/2863] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:30:32 2022] DELETE /identity/v3/projects/00e2faff80764bc0ad441cb80a8051fc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:32.792340 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.792632 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.792910 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.810814 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:32.812040 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:32.873401 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1438/2864] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:32.881879 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.882257 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.883155 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.902707 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:32.903829 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:32.912869 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1427/2865] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:32.917326 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:32.928088 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751d7c0-9a49-4e33-8d2a-5b6d16a245a6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:32.933235 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:32.933886 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.934200 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.934466 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] PATH_INFO: `/v3/domains` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.935227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80507) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 07 13:30:32.935531 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:32.937122 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:32.949407 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1439/2866] 10.222.0.61 () {58 vars in 1152 bytes} [Tue Jun 7 13:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 07 13:30:32.953835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:32.963387 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0ffca-191e-4ae4-a866-b8ec1e38d566 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:32.968101 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:32.968816 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:32.969111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:32.969399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:32.970113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:32.972087 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:32.998674 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1428/2867] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:30:32 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:33.002986 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:33.014275 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a71d1-5896-4305-8b65-59d8a9a0831c admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:33.073930 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:33.074611 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:33.074798 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:33.075103 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:33.075794 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:33.077773 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:33.097350 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1440/2868] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:30:33 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:33.102230 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:33.113932 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096059de-f754-47cd-978e-04bfe3ebbfe1 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:33.119478 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:33.120145 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:33.120484 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:33.120765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:33.121496 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:33.123280 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:33.136118 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1429/2869] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:33.140318 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:33.150737 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75100f2-50f4-4417-a204-c618e64d0b4e admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:33.156721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e801215-54c9-4750-9c8a-f04b8997d2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:33.156721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e801215-54c9-4750-9c8a-f04b8997d2de admin admin] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:33.156721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e801215-54c9-4750-9c8a-f04b8997d2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:33.156721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e801215-54c9-4750-9c8a-f04b8997d2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:33.157370 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e801215-54c9-4750-9c8a-f04b8997d2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:33.159027 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e801215-54c9-4750-9c8a-f04b8997d2de admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:33.170065 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1441/2870] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:33.174244 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:33.184898 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae162de-4624-4281-bd26-1964f4558cbb admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:33.189653 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:33.190523 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:33.190808 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:33.191111 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] PATH_INFO: `/v3/projects/3347761872914cbbb350a639eece5f9f/users/a4b564431e6e4f03ba125f749218777a/roles/2bbf1931a8df4f6da54f4d30d99aa430` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:33.206619 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3347761872914cbbb350a639eece5f9f, user_id=a4b564431e6e4f03ba125f749218777a, role_id=2bbf1931a8df4f6da54f4d30d99aa430)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:33.209325 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:33.222527 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1430/2871] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:33 2022] PUT /identity/v3/projects/3347761872914cbbb350a639eece5f9f/users/a4b564431e6e4f03ba125f749218777a/roles/2bbf1931a8df4f6da54f4d30d99aa430 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:33.228227 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64628e62-8aca-4c82-b249-44688e2b507a None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:33.228513 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64628e62-8aca-4c82-b249-44688e2b507a None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:33.228786 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64628e62-8aca-4c82-b249-44688e2b507a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:33.247634 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-64628e62-8aca-4c82-b249-44688e2b507a None None] MFA Rules not processed for user `05d0278bf67f4654af88f4f7c25092c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:33.248848 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-64628e62-8aca-4c82-b249-44688e2b507a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:33.291233 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:33.311775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1442/2872] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:30:33.321415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:33.323194 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:33.381017 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0af9ce-e06a-4ad4-b11d-243cb8ddf0d3 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:33.384830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:33.385466 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:33.385746 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:33.386020 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:33.395036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:33.396451 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:33.406259 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1431/2873] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:37.800612 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:37.801150 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:37.801280 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:37.843796 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] MFA Rules not processed for user `a4b564431e6e4f03ba125f749218777a`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:37.845601 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:37.941024 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1443/2874] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:37.943761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:37.967274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca606ace-a34a-44ae-b974-97daa80de20d service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:37.990877 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:37.991113 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:37.991503 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:37.991848 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:37.992623 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:37.995403 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:38.029110 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1432/2875] 10.222.0.61 () {60 vars in 1152 bytes} [Tue Jun 7 13:30:37 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 07 13:30:38.033306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:38.056775 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7083045-80d8-4187-b557-c26c7e82ccb6 None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:38.129101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:38.129978 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:38.130323 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:38.130663 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] PATH_INFO: `/v3/users` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:38.131497 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:38.134742 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:38.166874 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1444/2876] 10.222.0.61 () {60 vars in 1143 bytes} [Tue Jun 7 13:30:38 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 07 13:30:38.169434 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:38.182449 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afebad6a-70b5-4974-8de6-be136c18eb23 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:38.202210 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:38.202950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:38.204178 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:38.204409 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:38.205306 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:38.206817 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:38.221534 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1433/2877] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:38.225319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:38.244687 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04155f8d-dcb7-42fe-8624-ed17b5b80955 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:38.255257 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:38.258151 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:38.258434 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:38.258709 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] PATH_INFO: `/v3/projects/d15b8f7b896941a08349f1e6259219a0/users/ab2316577bce47928b9b0cfc2db125a8/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:38.277404 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15b8f7b896941a08349f1e6259219a0, user_id=ab2316577bce47928b9b0cfc2db125a8, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:38.280425 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:38.302937 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1445/2878] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:38 2022] PUT /identity/v3/projects/d15b8f7b896941a08349f1e6259219a0/users/ab2316577bce47928b9b0cfc2db125a8/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:38.309100 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:38.322494 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658f954-c455-461a-b78a-5d1cfe237b03 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:38.374546 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:38.426797 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa0036-70a5-4a5a-aea3-77e05c4bb396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:38.430222 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa0036-70a5-4a5a-aea3-77e05c4bb396 admin admin] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:38.431411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa0036-70a5-4a5a-aea3-77e05c4bb396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:38.431411 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa0036-70a5-4a5a-aea3-77e05c4bb396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:38.432011 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa0036-70a5-4a5a-aea3-77e05c4bb396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:38.433882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa0036-70a5-4a5a-aea3-77e05c4bb396 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:38.450172 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1434/2879] 10.222.0.61 () {58 vars in 1121 bytes} [Tue Jun 7 13:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 07 13:30:38.454237 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:38.466300 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c7546-54cc-4403-bb97-8c68fc1e2dfe admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:38.473924 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:38.474878 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:38.475161 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:38.475433 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] PATH_INFO: `/v3/projects/d15b8f7b896941a08349f1e6259219a0/users/ab2316577bce47928b9b0cfc2db125a8/roles/4ad1dfee8ff946ffb420d546aad553ab` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:38.478378 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15b8f7b896941a08349f1e6259219a0, user_id=ab2316577bce47928b9b0cfc2db125a8, role_id=4ad1dfee8ff946ffb420d546aad553ab)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:38.481553 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:38.494518 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1446/2880] 10.222.0.61 () {60 vars in 1482 bytes} [Tue Jun 7 13:30:38 2022] PUT /identity/v3/projects/d15b8f7b896941a08349f1e6259219a0/users/ab2316577bce47928b9b0cfc2db125a8/roles/4ad1dfee8ff946ffb420d546aad553ab => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:30:43.815585 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba04f3d-b40c-4737-b6de-7a5d7adcc6eb None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:43.816596 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba04f3d-b40c-4737-b6de-7a5d7adcc6eb None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:43.816769 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba04f3d-b40c-4737-b6de-7a5d7adcc6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:43.849755 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-4ba04f3d-b40c-4737-b6de-7a5d7adcc6eb None None] MFA Rules not processed for user `ab2316577bce47928b9b0cfc2db125a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:43.851416 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-4ba04f3d-b40c-4737-b6de-7a5d7adcc6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:43.951831 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1435/2881] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:43.970284 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:30:43.973691 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:44.016672 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:44.109179 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12295f0d-b2eb-409d-a241-92bc3157f994 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:30:44.117717 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:30:44.118382 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:44.118675 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:44.119278 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:44.201424 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:30:44.204153 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:30:44.219494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1447/2882] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:30:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:30:46.377516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:30:46.379083 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:30:46.380499 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:30:46.402471 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] MFA Rules not processed for user `a4b564431e6e4f03ba125f749218777a`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:30:46.404405 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:30:46.412131 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1436/2883] 10.222.0.61 () {58 vars in 957 bytes} [Tue Jun 7 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:30:54.214142 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=56 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:30:56.397252 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=51 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:31:03.081308 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:31:03.082945 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:03.144544 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb8b90-1e12-4c31-a0f3-2d5a94247558 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:31:03.148273 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:31:03.148922 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:31:03.149202 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:31:03.149496 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:31:03.151471 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:03.162992 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:31:03.166046 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:31:03.178177 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1448/2884] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:31:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:31:03.903505 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:31:03.905274 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:03.981354 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66981844-a0db-42df-b8ab-258c73acf1b8 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:31:03.986577 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:31:03.987260 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:31:03.987554 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:31:03.987835 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:31:03.989725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:04.001949 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:31:04.003600 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:31:04.014795 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1437/2885] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:31:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:31:13.173373 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=26 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:31:14.010330 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=32 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:31:16.892631 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:31:16.895243 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:16.916866 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323fed1-6872-49b3-bec3-8f431f695587 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:31:16.923784 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:31:16.925181 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:31:16.925476 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:31:16.926033 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:31:16.929949 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:16.947351 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:31:16.949306 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:31:16.972032 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1449/2886] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:31:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:31:19.530003 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:31:19.539421 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0d5c8-a04c-4b7c-9b6b-233be9b57f35 service placement] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:31:19.542957 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:31:19.543597 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:31:19.543880 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:31:19.544158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:31:19.553507 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:31:19.554975 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:31:19.568262 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1438/2887] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:31:21.143642 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:31:21.154917 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e54af-b3fe-41bc-be24-936385b170b3 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:31:21.160728 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:31:21.161371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:31:21.161701 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:31:21.161976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:31:21.163770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:31:21.176868 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:31:21.178438 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:31:21.189820 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1450/2888] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:31:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:31:29.563583 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=6 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:31:31.185248 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=12 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:32:16.458334 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:32:16.459852 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:32:16.499406 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeea135-36ae-4f49-8bbf-d74c383f9bd1 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:32:16.504614 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:32:16.505272 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:32:16.505555 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:32:16.505830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:32:16.507671 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:32:16.520639 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:32:16.522237 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:32:16.534202 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1439/2889] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:32:26.529516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=15 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:33:19.758385 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:33:19.774748 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14898f8-e925-4347-95f0-613a57149c98 service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:33:19.781165 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:33:19.782770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:19.782770 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:19.783371 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:19.785627 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:33:19.808209 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:33:19.809722 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:33:19.822503 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1451/2890] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:33:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:33:21.016036 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:33:21.025432 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dcd50-78db-4a72-b150-bdef0d9ce30a service placement] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:33:21.029033 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:33:21.029673 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:21.029950 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:21.030224 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:21.031974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:33:21.045612 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:33:21.047047 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd80f44-0e92-4c35-8bee-76dda0a00850 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:33:21.058541 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1440/2891] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:33:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:33:29.816279 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=6 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:33:31.053018 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=6 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:33:34.162319 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:33:34.171422 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f9a5-d7f1-4d2b-938b-8e0012c1a5f0 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:33:34.175101 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:33:34.175750 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.176034 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.176312 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.178098 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:33:34.189069 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:33:34.190510 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1dd00-6b53-401c-9958-e5bd7272a55c service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:33:34.199962 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1452/2892] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:33:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:33:34.314446 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5562f4-1452-4a0e-ba49-fb8751f84728 None None] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.315065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5562f4-1452-4a0e-ba49-fb8751f84728 None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.315065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5562f4-1452-4a0e-ba49-fb8751f84728 None None] PATH_INFO: `/` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.316487 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1441/2893] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:33:34.324374 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7839ea4-5f75-4b7f-b0b5-250744fb12ca None None] REQUEST_METHOD: `POST` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.324659 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7839ea4-5f75-4b7f-b0b5-250744fb12ca None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.324934 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7839ea4-5f75-4b7f-b0b5-250744fb12ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.379926 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.auth.core [None req-a7839ea4-5f75-4b7f-b0b5-250744fb12ca None None] MFA Rules not processed for user `6370d72aa28c4cce952ca67ce5452cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80507) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:33:34.381263 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-a7839ea4-5f75-4b7f-b0b5-250744fb12ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:33:34.389316 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1453/2894] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:33:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 07 13:33:34.400431 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5562f4-1452-4a0e-ba49-fb8751f84728 None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:33:34.419134 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5562f4-1452-4a0e-ba49-fb8751f84728 None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:33:34.424782 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35d5f6-e21b-4555-b6ed-505015b6afb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:33:34.426010 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35d5f6-e21b-4555-b6ed-505015b6afb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.426455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35d5f6-e21b-4555-b6ed-505015b6afb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.426913 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35d5f6-e21b-4555-b6ed-505015b6afb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.438974 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35d5f6-e21b-4555-b6ed-505015b6afb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:33:34.440713 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35d5f6-e21b-4555-b6ed-505015b6afb9 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:33:34.452415 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1442/2895] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:33:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 07 13:33:34.544734 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9791c8-69d4-404f-801c-f3bc2eba2e7d None None] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.545021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9791c8-69d4-404f-801c-f3bc2eba2e7d None None] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.545305 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9791c8-69d4-404f-801c-f3bc2eba2e7d None None] PATH_INFO: `/` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.546705 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1454/2896] 10.222.0.61 () {54 vars in 921 bytes} [Tue Jun 7 13:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 07 13:33:34.552594 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] REQUEST_METHOD: `POST` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.552882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.553156 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.592310 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.auth.core [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] MFA Rules not processed for user `e56bcda01a104f5795f962662cadcd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80506) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 07 13:33:34.593519 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:33:34.603697 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1443/2897] 10.222.0.61 () {58 vars in 1018 bytes} [Tue Jun 7 13:33:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 07 13:33:34.614255 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9791c8-69d4-404f-801c-f3bc2eba2e7d None None] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:33:34.626456 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9791c8-69d4-404f-801c-f3bc2eba2e7d None None] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:33:34.630260 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:33:34.630952 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:33:34.631240 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:33:34.631520 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:33:34.642342 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:33:34.643816 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:33:34.655337 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1455/2898] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:33:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:33:44.589756 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=15 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:33:44.650327 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=25 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:34:20.075575 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:34:20.086552 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1642d5-e7ec-44be-a53f-d3e4c95415ad None None] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:34:20.090158 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6370d72aa28c4cce952ca67ce5452cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:34:20.090810 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:34:20.091122 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:34:20.091427 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:34:20.093254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:34:20.107255 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:34:20.108847 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:34:20.117590 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1444/2899] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:34:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:34:30.117516 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=6 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:35:08.332264 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:35:08.344872 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342ad7d-2894-424b-92e6-1c45b5bb1f48 service neutron] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:35:08.352903 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56bcda01a104f5795f962662cadcd83', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:35:08.353805 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:35:08.354346 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:35:08.354625 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:35:08.356651 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:35:08.369859 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:35:08.371511 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:35:08.384131 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1456/2900] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:35:16.362455 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:35:16.391854 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930eae6-1433-4ac7-921a-8241d4160218 service neutron] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:35:16.398012 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:35:16.399069 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] REQUEST_METHOD: `GET` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:35:16.399198 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:35:16.399399 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:35:16.401297 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:35:16.427620 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:35:16.431065 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:35:16.459457 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1445/2901] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:35:18.379048 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=6 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:35:19.977896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:35:20.041772 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e4c2-3812-49b6-afbc-a1de731b5bab service nova] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:35:20.046392 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:35:20.047286 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:35:20.047565 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:35:20.047833 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] PATH_INFO: `/v3/users/a4b564431e6e4f03ba125f749218777a` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:35:20.049041 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b564431e6e4f03ba125f749218777a)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:35:20.051113 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:35:20.100116 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b564431e6e4f03ba125f749218777a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.101002 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b564431e6e4f03ba125f749218777a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.108021 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b564431e6e4f03ba125f749218777a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.108649 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b564431e6e4f03ba125f749218777a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.126721 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b564431e6e4f03ba125f749218777a'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.142524 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1457/2902] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:35:19 2022] DELETE /identity/v3/users/a4b564431e6e4f03ba125f749218777a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:35:20.335786 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:35:20.351578 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35a71a-74f7-4e4e-9eba-ea6dac439744 service placement] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:35:20.417459 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:35:20.418404 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:35:20.418943 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:35:20.419227 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] PATH_INFO: `/v3/projects/3347761872914cbbb350a639eece5f9f` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:35:20.420725 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3347761872914cbbb350a639eece5f9f)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:35:20.422784 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:35:20.479830 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3347761872914cbbb350a639eece5f9f', 'request_id': 'req-829aa9d7-f47d-4788-84c4-23cfb8b71200'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.487254 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3347761872914cbbb350a639eece5f9f', 'request_id': 'req-829aa9d7-f47d-4788-84c4-23cfb8b71200'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.497424 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3347761872914cbbb350a639eece5f9f', 'request_id': 'req-829aa9d7-f47d-4788-84c4-23cfb8b71200'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.509006 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] The token cache is being invalidate because project 3347761872914cbbb350a639eece5f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:35:20.509878 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.510089 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.511789 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1446/2903] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:35:20 2022] DELETE /identity/v3/projects/3347761872914cbbb350a639eece5f9f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:35:20.516692 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:35:20.518303 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:35:20.531940 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea24d0c-e643-45ee-b592-672933a943f2 admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:35:20.589990 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:35:20.590895 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:35:20.591182 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:35:20.591458 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] PATH_INFO: `/v3/users/ab2316577bce47928b9b0cfc2db125a8` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:35:20.592667 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2316577bce47928b9b0cfc2db125a8)` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:35:20.594595 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:35:20.619674 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:35:20.654616 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2316577bce47928b9b0cfc2db125a8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.655746 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2316577bce47928b9b0cfc2db125a8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.662976 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2316577bce47928b9b0cfc2db125a8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.663630 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2316577bce47928b9b0cfc2db125a8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.675718 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.notifications [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2316577bce47928b9b0cfc2db125a8'} {{(pid=80507) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:20.689744 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1458/2904] 10.222.0.61 () {58 vars in 1223 bytes} [Tue Jun 7 13:35:20 2022] DELETE /identity/v3/users/ab2316577bce47928b9b0cfc2db125a8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:35:20.869141 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Authenticating user token {{(pid=80506) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:35:20.870668 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.fernet_utils [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80506) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:35:20.892866 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aa9d7-f47d-4788-84c4-23cfb8b71200 admin admin] Validating token access rules against request {{(pid=80506) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:35:20.951770 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d0278bf67f4654af88f4f7c25092c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9260a652cb24f75867436268df3aa69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80506) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:35:20.952660 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:35:20.953313 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] SCRIPT_NAME: `/identity` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:35:20.953598 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] PATH_INFO: `/v3/projects/d15b8f7b896941a08349f1e6259219a0` {{(pid=80506) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:35:20.954814 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15b8f7b896941a08349f1e6259219a0)` {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:35:20.956761 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] RBAC: Authorization granted {{(pid=80506) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:35:20.981128 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:35:21.009453 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15b8f7b896941a08349f1e6259219a0', 'request_id': 'req-2a081fd5-9425-47cf-823c-3d7705bf078e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:21.015882 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15b8f7b896941a08349f1e6259219a0', 'request_id': 'req-2a081fd5-9425-47cf-823c-3d7705bf078e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:21.024139 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15b8f7b896941a08349f1e6259219a0', 'request_id': 'req-2a081fd5-9425-47cf-823c-3d7705bf078e'} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:21.034745 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] The token cache is being invalidate because project d15b8f7b896941a08349f1e6259219a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80506) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 07 13:35:21.035064 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:21.035716 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG keystone.notifications [None req-2a081fd5-9425-47cf-823c-3d7705bf078e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80506) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 07 13:35:21.037765 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: [pid: 80506|app: 0|req: 1447/2905] 10.222.0.61 () {58 vars in 1232 bytes} [Tue Jun 7 13:35:20 2022] DELETE /identity/v3/projects/d15b8f7b896941a08349f1e6259219a0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 07 13:35:30.686205 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:35:31.031735 dv-cinder-844601-3-iscsi devstack@keystone.service[80506]: DEBUG dbcounter [-] [80506] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80506) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 07 13:37:21.712012 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Authenticating user token {{(pid=80507) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 07 13:37:21.713445 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:37:21.783266 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929c1f3-9384-421d-951a-1bf33e00ac3d admin admin] Validating token access rules against request {{(pid=80507) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 07 13:37:21.844561 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '702fa2f56dab462ea0c644ddb516ae8c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34114a5ed9d147c585097748da3c008b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80507) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 07 13:37:21.845211 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] REQUEST_METHOD: `GET` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 07 13:37:21.845494 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] SCRIPT_NAME: `/identity` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 07 13:37:21.845768 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80507) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 07 13:37:21.847555 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.fernet_utils [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80507) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 07 13:37:21.859896 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 07 13:37:21.861485 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99858d0a-b5b0-4f7d-a98c-2b35ba9c0e99 service placement] RBAC: Authorization granted {{(pid=80507) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 07 13:37:21.872552 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: [pid: 80507|app: 0|req: 1459/2906] 10.222.0.61 () {60 vars in 1365 bytes} [Tue Jun 7 13:37:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 07 13:37:31.867873 dv-cinder-844601-3-iscsi devstack@keystone.service[80507]: DEBUG dbcounter [-] [80507] Writing DB stats keystone:SELECT=58 {{(pid=80507) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}